
<ColorModeImg
    lightSrc="/images/posts/immich/overview.png"
    alt="Google Photos"
/>

# Introduction

This post explains why and how I moved about 1TB of photos and videos from cloud services to a local photo server. My main requirement was simple: sync media directly from mobile devices and access the original files quickly, especially when I am at home.

# Overview

### Pros

1. Access speed: Most of the time, I browse my media at home. A local server lets me access photos and videos quickly at full quality.
2. Privacy and control: I do not want to depend on a third party for my personal media. With a local setup, I control where the data lives and how it is accessed, assuming the setup is secured properly.

### Cons

- Technical complexity: Unless you enjoy building and maintaining your own services, this is more complex than uploading everything to cloud storage. The long-term cost can be lower if the hardware runs for five or six years, but there is still an upfront cost and ongoing electricity usage to consider.
- Risk of data loss: Without a good backup plan, such as the [3-2-1 backup strategy](https://www.backblaze.com/blog/the-3-2-1-backup-strategy/), you are responsible for the risk of losing your media.

# How it works

### Hardware

<NetworkAnimation />

I had an old WD external HDD with most of my older photos and videos backed up, so I plugged it into a Lenovo Tiny PC with 8GB of RAM and an 8th-generation i5 processor. Initially, this was just for experimenting with Immich features.

For networking, I use a Linksys MR5500 router with OpenWRT firmware and a Cloudflare Tunnel to access the server from outside my home network.

The Lenovo Tiny PC does not have a GPU, so I use my personal laptop with a 5060 mobile GPU to run the Immich machine learning container for face recognition and other GPU-accelerated features.

### Software

The core setup is Immich running in Docker, with Nginx in front of it for HTTPS and password protection. The Immich machine learning container runs separately on my laptop and connects back to the server.

### Setting up the server

I installed the Immich server on the Lenovo Tiny PC with Docker and configured it to use a folder on the external HDD as an external media library.

I configured Nginx to enforce HTTPS and password protection when accessing Immich through the Cloudflare Tunnel.

Finally, I updated the Immich Docker Compose file to use the machine learning container running on my personal laptop.

### Security

I use WPA3 for WiFi security and keep the router firmware up to date. I also use strong passwords for Nginx authentication and Cloudflare Tunnel access.

There is no port forwarding, no public IP exposure, and no plain HTTP access to the server.

<img
  src="/images/posts/immich/immich.png"
  alt="Immich"
  width="50px"
/>


From Google Photos ☁ to Immich 🖼️



<img src="/images/posts/react-state-init/thumb.png/" width="400px"/>

# State initializer function

One of the most commonly used React hooks is `useState`. However, it's not widely known that `useState` can accept a function as an initializer. This initializer function is called only once, when the React component initially mounts.


```jsx focus=9:18 mark=11[26:37]
const initializer = () => {
  console.log("Initializer function called");
  let initialCount = 0;
  for (let i = 0; i < 10 ** 3; i++) {
    initialCount += 1;
  }
  return initialCount;
};
export default function App() {
  const [count, setCount] = useState(initializer);

  return (
    <div>
      <h1>Hello Initializer Function!</h1>
      <h3>Count: {count}</h3>
      <button onClick={() => setCount(count + 1)}>Re-Render</button>
    </div>
  );
}
```
The initializer function must be pure, meaning it should not cause any side effects and should always return the same value for the same input. Additionally, the initializer function cannot return a promise; it must return the initial state value synchronously.

# Benefits of using initializer function

Main benefit is performance optimization. If your initialization logic is computation expensive, Moving it to an initializer function will help to improve the performance.

# Demo

[CodeSandbox](https://codesandbox.io/p/sandbox/react-state-initialization-hvtzv5?file=%2Fsrc%2FApp.tsx%3A12%2C1-13%2C1)



<Spack filename="/App.js">
{`
import React, { useState } from "react";

const initializer = () => {
  console.log("Initializer function called");
  let initialCount = 0;
  for (let i = 0; i < 10 ** 3; i++) {
    initialCount += 1;
  }
  return initialCount;
};
export default function App() {
  const [count, setCount] = useState(initializer);

  return (
    <div>
      <h1>Hello Initializer Function!</h1>
      <h3>Count: {count}</h3>
      <button onClick={() => setCount(count + 1)}>Re-Render</button>
    </div>
  );
}
`}
</Spack>

React State initializer function


## Use Case

Let's assume that you need additional information for a list of items in an array. For example, you may have initially loaded a set of users from your API and now want to fetch their metadata through the Gravatar API.

In this case, you will need to loop through the list of users and make individual API calls to the Gravatar API to retrieve their metadata. In this article, we will discuss how to efficiently retrieve this secondary data while keeping the page interactive for the users.

## The worst case

A for loop with an await statement is the least efficient method for fetching data by iterating through a list of items.
In fact there is an [eslint rule](https://eslint.org/docs/latest/rules/no-await-in-loop) to mitigate this scenario.
But why it's bad ?

Let's take the following example.

```jsx focus=9:18 mark=11[26:37]
export default function Worst() {
    const [users, setUsers] = useState(null);
    const [loading, setLoading] = useState(true);

    useEffect(() => {
        (async () => {
            const allUsers = [];
            for (const id of userIDs) {
                const response = await fetch(
                    `${apiEndpoint}/${id}?mocky-delay=${rndInt()}ms`
                );
                const data = response.json();
                allUsers.push(data);
            }
            setUsers(allUsers);
            setLoading(false);
        })();
    }, []);

    //...
}
```

The problem lies mainly in the `await fetch` expression, Javascript pause the execution at this line and wait for the promise fulfilment and rejection.
This creates the so called "waterfall" effect. In other words this expression causes API calls to wait for previous call resolve/reject before making the next call.
This sequencing effect can be seen in the network panel if you open the browser Developer Console.

![Network Waterfall](/images/posts/fetch_in_react/waterfall.png)

As shown above, The execution will wait at each API call. We will be setting the completed result after iterating through the entier array.
Since `setUsers` happens after this, Users will preseve an combined latancy of individual API requests.

## Commonly used pattern

To mitigate this issue, We could iterate through the array and invoke the API. Tricky part here is we are not `await`ing here.
and we collect all the promise objects returned from the fetch API into an array. and then use [`Promise.all` API](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all) API to wait till all of the promises to settle.

```jsx focus=6:13 mark=9[5:15]
export default function Normal() {
    const [users, setUsers] = useState(null);
    const [loading, setLoading] = useState(true);

    useEffect(() => {
        const promisedUsers = userIDs.map((id) =>
            fetch(apiEndpoint).then((response) => response.json())
        );
        Promise.all(promisedUsers).then((results) => {
            setLoading(false);
            timing.current.end = Date.now();
            setUsers(results);
        });
    }, []);

    // . . .
}
```

Now API calls ara sent in paralle, and there is no inter-dependency for making the call.

![Network Waterfall](/images/posts/fetch_in_react/slightly_improved.png)

But there is a catch here. The `Promise.all` API will not run the call back until

> all of the input's promises fulfill

[~Source](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise/all)

So, Even though this aproach has improved the latancy a lot, Still the time to render the data depends on the slowest API call in the pool.
User will preseve a latancy of the slowest API call.

We could further improve the latancy, Let's see how.

## Render with data

In this approach, we render(update the state) then and there when the data arrive, So users will see the data as soon as they are arrived
The trick here is, We [chain a success callback](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Guide/Using_promises#chaining) for each fetch API response promise,
which will eventually update the state when the callback function is executed.

```jsx focus=6:17 mark=9[9:25]
export default function Best() {
    const [users, setUsers] =
        (useState < null) | ([{ first_name: string }] > null);
    const [loading, setLoading] = useState(true);

    useEffect(() => {
        userIDs.map((id) =>
            fetch(apiEndpoint)
                .then((response) => response.json())
                .then((data) => {
                    setLoading(false);
                    if (!timing.current.end) {
                        timing.current.end = Date.now();
                    }
                    setUsers((prevUsers) =>
                        prevUsers ? [...prevUsers, data] : [data]
                    );
                })
        );
    }, []);

    // . . .
}
```

With this mechanisum, Each API call will inidividually update the state with their data. Other change is using the [React state updater function](https://react.dev/reference/react/useState#updating-state-based-on-the-previous-state) instead of directly passing the value.

![Network Waterfall](/images/posts/fetch_in_react/improved.png)

Want to see the code and a demo?

## Source and demo

# [Source in CodeSandbox](https://codesandbox.io/s/react-parallel-data-handling-v72j3j)

# [Demo App](https://csb-v72j3j.vercel.app/)


React fetch in a loop


## Coming soon

This article will be based on [this source code](https://codesandbox.io/s/react-hooks-how-it-works-8k2ju) and will provide an in-depth explanation of how React hooks work.

# Credits

I used [@swyx](https://twitter.com/swyx)'s article [Getting Closure on React Hooks](https://www.swyx.io/hooks) as a guide for my implementation.


How React hooks work


## About

In this post, I will explain how to use the Sandpack editor with MDX files in NextJS app.

## Prepare

I'm using [next-mdx-remote](https://github.com/hashicorp/next-mdx-remote) to render `MDX` files and [@codesandbox/sandpack-react](https://github.com/codesandbox/sandpack) package to add `Sandpack` editor.
To install them run

```
npm i next-mdx-remote @codesandbox/sandpack-react
```

## Configurations

Custom MDX component to render `Sandpack` editor
Write a custom Sandpack component with the pre-configured theme, Sandpack options etc,
Following is the `CustomSandpack` component I built to use in MDX content

```jsx
import { Sandpack } from "@codesandbox/sandpack-react";
import { useTheme } from "@mui/material";

type CustomSandpackProps = {
  template: any;
  filename: string;
  children: string;
};
const CustomSandpack = (props: CustomSandpackProps) => {
  const theme = useTheme();
  const { children, filename } = props;
  return (
    <Sandpack
      template="react-ts"
      theme={theme.palette.mode}
      files={{
        [filename]: { code: children, active: true },
      }}
      options={{
        showLineNumbers: true,
        showInlineErrors: true,
        showTabs: false,
        closableTabs: false,
      }}
    />
  );
};

export default CustomSandpack;
```

Inject the `CustomSandpack` component to the MDX context
Next, import the `CustomSandpack` component to the `[slug].tsx` where the NextJS dynamic routing happens and pass the `CustomSandpack` component to the `MDXRemote` via the `components` prop.

```jsx

const components: any = {
    // It also works with dynamically-imported components, which is especially
    // useful for conditionally loading components for certain routes.
    // See the notes in README.md for more details.
    Spack: dynamic(import("../../src/components/blog/CustomSandpack")),
};
.
.
.
                    <MDXRemote {...mdxSource} components={components} />
```

Use the `<Sandpack>` element in MDX

Finally, You can use the `Sandpack` custom component in MDX content as below

```mdx
<Spack filename="/App.js">
    {`export default function App() {
        return <h1>Hello Sandpack from Knnect</h1>
        }`}
</Spack>
```

## Result

> Try editing the code below and see the result

<i>Try out Dark vs Light modes</i>

<Spack filename="/App.js">
    {`export default function App() {
  return <h1>Hello Sandpack from Knnect</h1>
}
`}
</Spack>


Sandpack with NextJS


# Preface

If you are already familiar with the WSO2 API Manager product and know about its recent UI revamping, You can skip this section and directly jump into the Introduction.

About [a year ago](https://github.com/wso2/product-apim/releases/tag/v3.0.0) WSO2 API Manager team released the next major version of API Management platform 3.0.0, It was a major version change from its predecessor 2.x. As of the major version change we have done a major UI revamp in the product as well. In version 3.0.0 we removed the JaggeryJS based Publisher & Store portals(Webapps) and built new Publisher & Devportals apps from the ground up using the ReactJS library. They are 100% client-side rendering single-page applications(SPA) now. You can find more information about this major revamp in [this article](https://medium.com/@tmkasun/wso2-api-manager-new-look-27a186bc83d5).
About 5 months later we released the API Manager 3.1.0 version with many improvements,bug fixes and of course new features. You can find [What’s new in API Manager 3.1.0 portals](https://medium.com/@tmkasun/what-is-new-in-wso2-api-manager-3-1-0-5d37cea741cc) in this article.
And today we are looking at WSO2 API Manager version [3.2.0](https://github.com/wso2/product-apim/releases/tag/v3.2.0), which is the latest version of the API Manager 3.x family (and could be the last of 3.x as well). And as we all expect, 3.2.0 also has many bug fixes and improvements including UI customization improvements as well.
So that brings us to the topic of this article _Customizing WSO2 API Manager React portals_, without further ado let’s get started!

# Introduction

From API Manager 3.2.0 onwards all 3 portals in the API Manager server are React-based single-page applications. In this article, I will use the Publisher portal to explain the customization mechanism but it's the same for the other 2 portals as well. So the structure of this article is, First I will explain the React portals structure, then

# Structure

First, let’s see how the portal’s source code is organized. The Source of the publisher portal resides in the [carbon-apimgt](https://github.com/wso2/carbon-apimgt/tree/master/features/apimgt/org.wso2.carbon.apimgt.publisher.feature/src/main/resources/publisher) GitHub repository. And when building the product API Manager this source gets copied into

```
<API_MANAGER_ROOT>/repository/deployment/server/jaggeryapps/publisher/
```

Directory.
Source code in both these locations are same for a particular version and we ship the React apps source code along with the distribution for the customization purpose. Otherwise, You will not need to react application source code in the run time.
Let’s look at how the source code is organized in WSO2 API Manager Publisher portal

![Source structure](/images/posts/apim_320_customization/WSO2-API-Manager-3-x-portals-structure.png)

This is the source code structure of the publisher portal. Out of these files and directories, We only need services, site, and jaggery.conf files at the runtime, the rest of the source content is for development time requirements, such as dependency management (package\*.json), code quality (eslint.\*), etc.

-   **override**

    This folder is used to organize the React component customizations. This is empty in the default pack(distribution).In short, This is where you put your customizations to override the default React UI components.

-   **services**

    This directory contains the utility services that are required to run the web portals. Even though the React UI portals are fully client-side rendered(CSR) apps, We still need services or proxies for some operations, Such as

    -   Authentication
    -   Reverse proxy
    -   Custom URLs

-   **site**

    This contains all the static files that are needed to run the portals in the browser. And in fact, this is the directory contains various static files from JS bundles, locale files, theme JSON to favicon. Inside the site, you can find

    ```
    <API_MANAGER_ROOT>/site/public/dist/
    ```

    this directory contains the generated ReactJS application bundles and their source map files.

-   **source**

    This directory contains the Javascript source codes. It contains both implementation and tests. None of the files in here are used in runtime (get loaded into the browser). Here we have split the implementation into two aspects, Data, and Components. Data contains pure javascript implementations of data handling (REST API innovation), Utility functions, and data models (User class). The purpose of these separations is to make the data handling elements shareable and usable in non- ReactJS based projects. And in the Components directory, we maintain the ReactJS Component implementations. It has the whole component hierarchy of the publisher portal.

-   **jaggery.conf**

    This is a configuration file used for web apps in Jaggery web server, It contains

    -   routing rules(urlMappings)
    -   security constraints
    -   welcomeFiles

    and other jaggery app-related configurations.

    If you are new to JaggeryJS, JaggeryJS is a javascript backend server that can render jaggery style **_.jag_** (like Pug) templates into HTML. Simply put, It’s like NodeJS + Express

-   **others**

    The rest of the files are runtime configurations for eslint, jest, webpack, npm. You would probably recognize them by their names.

# Getting started

The first step for customizing the portal app is to take a copy of the portal packages into your development environment. If you take a look at unzipped/exploded API Manager 3.2.0 product distribution(pack), You can find all 3 UI portals inside the

```
<APIM_3.2.0_ROOT_DIRECTORY>/repository/deployment/server/jaggeryapps/
```

![](/images/posts/apim_320_customization/goto.gif)

API Manager server contains the portal source codes for bootstrapping the customization work. But it is not intended to do the customizations inside the server directory itself.
So first, copy the entire **jaggeryapps** directory into a desired location. This action is important Because if you follow later steps (Download dependencies, Run production/development build) inside the server itself, it will cause a performance issue in web apps. This performance issue is mainly caused by the sheer number of files getting downloaded when installing the dependencies (because of the **node_modules**). So make sure to take a copy at the beginning of the development.

![](/images/posts/apim_320_customization/blog_customization.png)

Now let’s see how to begin the development work.

# Dependency management

API Manager portals use NPM for package management, Except for some of the internally shared components, Such as the API Test console. API Test console React component has shared between the developer portal and Publisher portal So, We have used Lerna for managing those local sharing or intra dependencies.

> **_NOTE:_** Lerna packages were introduced in WSO2 API Manager 3.2.0, If you are customizing a 3.x version below 3.2.0 (3.0.0 or 3.1.0), You don't need to run below lerna commands

Lerna helps to manage both internal and external public library dependencies by wiring up local dependencies with symbolic links and downloading the external dependencies through the npm public registry. Before integrating the Lerna, we used to run

```bash
npm ci
```

to download the dependencies. And now it’s

```bash
lerna bootstrap
```

When customizing the portals, You need to run **lerna bootstrap** from the portal’s parent directory. To make the things easy for the users, we have added an npm alias

```bash
npm run bootstrap
```

to run the Lerna bootstrap locally.
That is the internals of how package dependencies are managed. But it’s just 2 commands to download dependencies for all 3 apps(Admin, Publisher & Devportal).
First, run

```bash
npm install

```

from the root directory of your project where the lerna.config and package.json files are located

```
<PROJECT_ROOT>/
```

`NOTE`: In here **\<PROJECT_ROOT>** refers to the **jaggeryapps/** directory that you have copied in one of the early steps

Then run

```bash
npm run bootstrap
```

This will do both downloading and sym-linking local dependencies.

That’s it, Now all the dependent packages are in place, Let see how to build the portal project.

# Build process

API portals use Webpack for bundling the ReactJS and other Javascript dependencies into a Single Page App (SPA) and It follows a typical simple webpack build process. We use a simple webpack loader to integrate the customizations into the build.

The following diagram shows where the loader sits in the application build process.

![](/images/posts/apim_320_customization/WSO2-API-Managerbuild_pipe.png)

The loader checks for the files in the **/override** directory and if it found matching files to currently process original source files, then it overrides the content of the original file with the content of the customization and passes forward in the build pipeline. In this way, users can override any implementation in the original source ( in **/source**) by providing a custom implementation in the **/override** directory.
You can check the webpac.config.json file, To learn more about the plugins and loaders that are used in the project.

![](/images/posts/apim_320_customization/WSO2-API-build.png)

Another important point to note here is the production and development build commands. We have defined **build:dev** and **build:prod** npm aliases to execute development and production builds. For example, if you want to start/run a development build run the

```bash
npm run build:dev
```

command from the portals root directory where the **package.json** file is located\.In default distribution publisher portal’s **package.json** is located in

```
/publisher/
```

Directory.

**NOTE**

It’s recommended to take a copy of the portal project to a different location before working on the customizations (goto **First Step** section if you missed that :) )

Let’s see how this build process and source structure links in the customization process.

# Customizing React Components

I will use a simple scenario to explain how the customization works. Let’s assume you want to add a new input parameter to API create page.Where API developers can provide a Github repository URL as a reference for API consumers. So that API consumers or application developers who are interested in this API can explore the code in the Github repository and bootstrap their work.
Let’s see how the Publisher **API create** page UI looks like after making the customization.

![](/images/posts/apim_320_customization/Publisher-WSO2-APIM-create-api.png)

We are going to add the input field that is marked with the red rectangle.

### 1. Locating React component

First, you need to locate the file or React component that is rendering the segment that you want to customize.

-   The easiest way to find the React component is to use the [React dev tools](/images/posts/apim_320_customization/https://chrome.google.com/webstore/detail/react-developer-tools/fmkadmapgofadopljbjfkapdkoienihi). This is a tool developed by React team to help React developers to debug and profile react applications.You can install the extortion/plugin to the browser and use it.
    ![](/images/posts/apim_320_customization/react_dev_tools.png)
-   Another method is to use network trace, If the Interactions with your target component triggers any network calls , You can use the stack trace in the Browser's network inspector to locate the source component
    ![](/images/posts/apim_320_customization/nt_trace.png)
-   Or the other option is to set **Event Listener Breakpoints** from browser Source panel(in Chrome) or Debugger panel (in FireFox). You most probably need to black-box some irrelevant handlers multiple times, But you will get to the original source component at some point, So have some patient when following this method
    ![](/images/posts/apim_320_customization/source_debug.png)

### 2. Creating the custom React component

Now you know what are/is the component(s) that you want to modify.So in this sample scenario, We have identified that we need to customize the

```
<PROJECT_ROOT>/publisher/source/src/app/components/Apis/Create/Default/APICreateDefault.jsx
```

`NOTE`: In here **\<PROJECT_ROOT>** refers to the **jaggeryapps/publisher** directory that you have copied in one of the early steps

Next step is to implement the custom component in override directory, First create the source file that you want to customize with exact name and relative path (relative to \<PROJECT_ROOT>/source ) inside the **\<PROJECT_ROOT>/override** directory.
Following diagram shows the source code placement in the original source directory and the override directory.

![](/images/posts/apim_320_customization/blog_customization-Page-2.png)

To begin with you can copy the original implementation from **\<PROJECT_ROOT>/source/** directory and do the modifications accordingly, Only downside of this is , When you taking WUM updates, In the WUM updates WSO2 will ship the updated source code, so you will have to manually merge the bug fixes or changes that are done to the original source code.

After working on the customization in override directory, You can run the webpack build to build the portal applications with your customizations. As mentioned earlier you can use

```bash
npm run build:dev
```

for building the portals in development mode, This command will not exit after the initial build until you press Ctrl/Command + C (This is the standard \*nix way of signaling a process to abort). So if you create a symbolic link to public directory in WSO2 API manager server's publisher portal

i:e

```bash
ln -s <PROJECT_ROOT>/site/public/ <API_MANAGER_ROOT>/repository/deployment/server/jaggeryapps/publisher/site/public/
```

Then you can just refresh the page in the browser and see the changes, When a development build is running.

TODO: Add/attach diff override vs original source here

When you finish customizing the portal, You can run following command to trigger a webpack production build

```bash
npm run build:prod
```

This will generate minified and optimized JS bundles in

```
<PROJECT_ROOT>/site/public/
```

directory, and these bundles will contains the customizations you have put in **override** directory. Now you can copy and replace the

```
<API_MANAGER_ROOT>/repository/deployment/server/jaggeryapps/publisher/site/public/
```

in WSO2 API Manager server with the newly build artifacts to apply the customizations.

`NOTE`: Make sure to copy the entire **\<PROJECT_ROOT>/site/public/** directory. Because we are appending a hash value to bundle filenames to avoid stale [cache bundles](https://webpack.js.org/guides/caching/)

and that's it actually!

# Summary

In summary, WSO2 API Manager has ReactJS based web portals, and you can customize the UI by overriding the original component implementations. We have implemented a webpack loader to integrate the external customizations into the app bundles when building the portals. Users can provide their customizations in override directory and build the application to integrate customizations into the portal bundles.


How to customize WSO2 API Manager 3.x portals


<h2>Introduction</h2>
This post would be a continuation of my previous post on <a href="http://me.knnect.com/blog/getting-started-with-nvidia-jetson-tk1/">getting started with NVIDIA Jetson TK1</a> for machine learning and image analysis works and <a href="http://me.knnect.com/blog/how-to-connect-flir-lepton-module-to-nvidia-jetson-tk1/">connecting FLIR Lepton camera to Jetson</a> TK1.In this post, I will discuss analyzing and visualizing the thermal data which is fetched from FLIR Lepton IR camera. In my last post, about connecting lepton camera to NVIDIA Jetson board, I have concluded the discussion by providing some sample camera data frames and images generated from Lepton module.In this post, I will discuss the tools and technics which I have used for analyzing and visualizing the data.This post will mostly cover the areas of preprocessing the data before applying any statistical learning methods on the sample data.Since we need to have a good understanding of the data and its behaviors before we jump into machine learning or playing with the data.Unless we don't have a good understanding of the data which we are dealing with in all dimensions, It would be unfair to expect something magic done by machine on behalf of us.So all this is about understanding our data.

It's great to see that technology is improving in great pace in both areas of GPU computing and thermal imaging. By the time I have published my last post and now, NVIDIA has `<a href="http://nvidianews.nvidia.com/news/nvidia-jetson-tx2-enables-ai-at-the-edge">`released`</a>` its next generation Jetson development board Jetson TX2(on Tuesday, March 7, 2017)

`<a href="http://me.knnect.com/blog/wp-content/uploads/2017/04/Jetson-TX2-Fig-3.jpg"><img className="aligncenter wp-image-470 size-full" src="http://me.knnect.com/blog/wp-content/uploads/2017/04/Jetson-TX2-Fig-3.jpg" alt="NVIDIA Jetson TX2" width="617" height="615" />``</a>`

<p><em>NVIDIA Jetson TX2</em></p>
 

and also FLIR has announced the Lepton 3.0 `<a href="http://www.flir.com/home/news/details/?ID=83959">`release`</a>`(On April 12, 2017) to the OEM market. So all these improvements in thermal imaging and high-performance single board computing contribute to opening new possibilities in thermal image analysis and application of thermal image processing in machine learning.

&nbsp;

`<a href="http://me.knnect.com/blog/wp-content/uploads/2017/04/flir-lepton-3.jpg"><img className="aligncenter wp-image-478" src="http://me.knnect.com/blog/wp-content/uploads/2017/04/flir-lepton-3.jpg" alt="flir lepton 3" width="530" height="264" />``</a>`

#FLIR Lepton camera



<h2>Lepton Thermal Data Capturing and Processing</h2>
As I discussed in my previous <a href="http://me.knnect.com/blog/how-to-connect-flir-lepton-module-to-nvidia-jetson-tk1/">post</a> I'm using <a href="https://github.com/groupgets/pylepton">PyLepton</a> library to capture data frames from lepton camera.To start exploring the data captured from Lepton camera, Since at this stage, we don't use any learning task or CPU/GPU intensive task, I wrote a data service client using python to simply transfer the lepton data frames from Jetson TK1 to the desktop computer.

https://gist.github.com/f041a531cb6f99ffd46dd8b15d46c3de

<div>It uses the Tornado <strong>WebSocketHandler</strong> to send the data over web socket protocol.You can find the complete code <a href="https://github.com/tmkasun/PyLepton_With_Jetson">here</a>.</div>
<div>Once you get the raw frames from the camera, you will notice that it is not withing the range of normal(RGB or Grayscale) image pixel values(0-255), but it has a largely spread pixel depth with it. That is because in lepton camera it has a large pixel depth(14bit) so that it carries a large variation in the intensity of LWIR. Since it is hard to map such depth to visible image pixel depth which is 255 per pixel, Once you transform intensity values to image pixel range(Even with histogram equalization etc), You will be loosing lots of small variations in intensity levels and even will go blackout the regions which are far away from most frequent pixel density values. So I thought of plotting the raw intensity values on a 3D graph so that we can get an idea about the variations and distribution of the IR intensities throughout the frame.</div>
<div>When the data is plotted using plotly in 3D surface graph, It is much more clear to understand how the intensity levels are distributed over pixels.</div>
<div></div>
<h3 style={{textAlign: "center"}}>(<span style={{color: "#3366ff"}}>Click on the graph to load the Plotly module</span>)</h3>
<div>

`<a title="IR Intensity 3D graph" href="https://plot.ly/~tmkasun/22/?share_key=8qDkv9v7ypNEcjNhokkKHu" target="_blank" rel="noopener noreferrer"><img className="" src="https://plot.ly/~tmkasun/22.png?share_key=8qDkv9v7ypNEcjNhokkKHu" alt="IR Intensity 3D graph" width="1300" />``</a>`

<h2><script src="https://plot.ly/embed.js" async="" data-plotly="tmkasun:22"></script></h2>
</div>
<div>
We can use the Matplotlib library to plot the 3D and 2D graph from the lepton data frame, But <a href="https://github.com/plotly/plotly.py">Plotly</a> has a more interactive feature than <a href="https://github.com/matplotlib/matplotlib">Matplotlib</a> when it comes to plotting 3D graphs.Below is the output from Matplotlib 2D and 3D plot, here I have used the
<pre className="">cm.nipy_spectral</pre>
</div>
<div>Color map for mapping colors to different intensity levels in thermal data.In this way, we can clearly see the variations in the IR intensity far more better that the grayscale scale.And also 3D add another dimension to our view of thermal data.</div>
<h2><a href="http://me.knnect.com/blog/wp-content/uploads/2017/04/flir_matplotlib.png"><img className="aligncenter size-full wp-image-488" src="http://me.knnect.com/blog/wp-content/uploads/2017/04/flir_matplotlib.png" alt="flir_matplotlib" width="1470" height="818" /></a>Conclusion</h2>
In this post, I simply want to discuss about the tools and techniques which we can use to see the data in different angle and get more in-depth idea of the thermal data which we getting from lepton camera, And also after looking at the data in this way, it seems that even though Lepton having very fewer pixel count(60x80) compared to sophisticated visible range cameras we have today, It have a very large pixel depth giving even slightest changes in temperature variation in its field of view.So we could use this highly detailed information to identify the objects around us.In an upcoming blog post, I will be discussing such methods of identifying heat emitting objects such as movements of humans from the data captured from lepton camera.

## [![flir_matplotlib](http://me.knnect.com/blog/wp-content/uploads/2017/04/flir_matplotlib.png)](http://me.knnect.com/blog/wp-content/uploads/2017/04/flir_matplotlib.png)Conclusion

In this post, I simply want to discuss about the tools and techniques which we can use to see the data in different angle and get more in-depth idea of the thermal data which we getting from lepton camera, And also after looking at the data in this way, it seems that even though Lepton having very fewer pixel count(60x80) compared to sophisticated visible range cameras we have today, It have a very large pixel depth giving even slightest changes in temperature variation in its field of view.So we could use this highly detailed information to identify the objects around us.In an upcoming blog post, I will be discussing such methods of identifying heat emitting objects such as movements of humans from the data captured from lepton camera.You can find the source codes from all the above plotting algorithms in this `<a href="https://github.com/tmkasun/PyLepton_With_Jetson">`GitHub repo`</a>`.

[1] : http://me.knnect.com/blog/getting-started-with-nvidia-jetson-tk1/

[2] : http://me.knnect.com/blog/how-to-connect-flir-lepton-module-to-nvidia-jetson-tk1/

[3] : http://nvidianews.nvidia.com/news/nvidia-jetson-tx2-enables-ai-at-the-edge

[4] : https://github.com/tmkasun/PyLepton_With_Jetson


Visualizing FLIR Lepton Thermal data for statistical analysis


![Linksys WRT 1900 AC V2](/images/posts/openwrt_mwan3/linksys_wrt1900AC_v2.jpg)

# Network plan

My main requirement is to route the internet traffic to specific WAN connection conditionally depending on it's hostname. For example, If someone try to access `facebook.com`, `twitter.com` or `youtube.com` etc. I want those requests to be routed via a specific WAN (or internet router) and all the other request should routed through 2nd internet connection (WAN).

Second requirement was to monitor and collect stats about my home local network usage and persist them for later analysis. One of the motivational factor for collecting usage data was,there are rumors in our country that some Internet Service Providers(ISPs) charge for more data than what the user has actually consumed :)

Below is my network setup:

![Network Setup](/images/posts/openwrt_mwan3/network_draw.png)

Diagram sources: [lucid](https://lucid.app/lucidchart/invitations/accept/98104790-f585-4b6c-8e90-58e513880562) & [excalidraw](https://excalidraw.com/#room=587b683f2467ed94de73,rXcfIg77oPYxdlxKc32nvw)

# Hardware

I'm using [Linksys WRT 1900AC V2 (Cobra)](https://www.linksys.com/us/support-product?pid=01t80000003KdJ7AAK) router with [OpenWRT](https://openwrt.org/) latest firmware(v19.07.5). But you can use any OpenWRT compatible device to configure this. There are some OpenWRT like firmwares i:e [dd-wrt](https://dd-wrt.com/) & [🍅 tomato](https://advancedtomato.com/)

# OpenWRT Multiple WAN configuration

To begin with , You need to setup ssh on the OpenWRT router and access the router as root user `ssh -v root@192.168.1.1`. Most of the configurations can be done via the web UI but, It's easy to track the changes when configuring via command line, I used to take backups using `scp` and edit the config files in laptop and push them back to router via `scp`.

OpenWRT has a `mwan3` package which allows you to configure various rules and policies on WAN connections. I have use `mwan3` to configure the conditional load balancing/routing to achieve my above requirement.

To [install](https://openwrt.org/docs/guide-user/additional-software/managing_packages) the `mwan3` package use

```bash
opkg update
opkg install mwan3
```

After installing `mwan3` keep it aside and configure the [IPSet](https://wiki.archlinux.org/index.php/Ipset), and later we can refer the IPSet names and create `mwan3` config file at once.

To configure dynamic hostname base WAN routing you need to do 3 things

1.  create a ipset
    ```shell
    ipset -N twitter hash:ip
    ```
    in the above `twitter` is the name of ipset `hash:ip` is a template kind of thing, When router get it's first ping request.
    Following command will list all the ip sets
    ```shell
    ipset list
    ```
    To get information about a specific ip set
    ```shell
    ipset -L twitter
    ```
    Where `twitter` is the IP set name,
    If system has resolved IPs for the name , Then those IPs will be shown under `Members` parameter
    ```
    root@knnect:~# ipset -L twitter
    Name: twitter
    Type: hash:ip
    Revision: 4
    Header: family inet hashsize 1024 maxelem 65536
    Size in memory: 416
    References: 1
    Number of entries: 9
    Members:
    69.195.179.128
    69.195.182.128
    69.195.165.128
    104.244.42.2
    104.244.42.194
    104.244.42.66
    69.195.177.128
    209.237.193.128
    104.244.42.130
    ```
    Important thing to note here is, When you run above command ip set will be added to the memory, and they don't get persisted. Hence restarting the router will clear all the added ipset. In other words ipset rules are saved in memory, upon a reboot they will be lost.
    Because of that I have created a startup script ([source code](https://gist.github.com/tmkasun/4f9e1fe6ed8254a115d2924f6b3b885d))
    ```
    /etc/init.d/autoipset
    ```
    to add all the ip sets I want when starting up the router. Instructions are given in above gist source on how to add the startup script. For more info about how to write init script in OpenWRT follow [this official doc](https://openwrt.org/docs/techref/initscripts)
2.  Update `dnsmasq.conf` with ipset name and dns name

    > What is [dnsmasq](https://openwrt.org/docs/guide-user/base-system/dhcp.dnsmasq)

    Open the

    ```
    /etc/dnsmasq.conf
    ```

    in router, and add ipset entries as below

    ```
    ipset=/.twitter.com/twitter
    ```

    here, AFAIK `twitter` after the last slash is the ipset name, Following is a sample `dnsmasq.conf` configuration content

    ```shell
    # /etc/dnsmasq.conf
    # Change the following lines if you want dnsmasq to serve SRV
    # records.
    # You may add multiple srv-host lines.
    # The fields are <name>,<target>,<port>,<priority>,<weight>

    # A SRV record sending LDAP for the example.com domain to
    # ldapserver.example.com port 289
    #srv-host=_ldap._tcp.example.com,ldapserver.example.com,389

    ipset=/.facebook.com/facebook
    ipset=/.instagram.com/instagram
    ipset=/.fbcdn.net/fbcdn
    ipset=/.twitter.com/twitter
    ipset=/.twimg.com/twimg
    ```

    after updating the config file run

    ```shell
    service dnsmasq restart
    ```

    to refresh the `dnsmasq`
    Note here:
    In my case default `dnsmasq` tool coming with the OpenWRT installation does not worked and I had to uninstall the pre-installed `dnsmasq` and install `dnsmasq-full` to get this work!

3.  Now IP sets are ready, So final thing is to configure the `mwan3`
    ![image](/images/posts/openwrt_mwan3/mwan3.png)

    If you are new to `mwan3` there is a really good [documentation in OpenWRT](https://openwrt.org/docs/guide-user/network/wan/multiwan/mwan3) docs,covering all the aspects.

    At this point, better to restart the router, and ping to those host name that you have configured above i:e

    ```shell
        ping twitter.com
    ```

    If you are curious about IPs and DNS, When you ping you will get one of following IPs at the time you execute the ping command

    ```shell
    whois -h whois.radb.net -- '-i origin AS13414' | grep ^route
    ```

    If you wonder how it resolve all the IPs , Google search about [ASN Lookup](https://www.ultratools.com/tools/asnInfoResult?domainName=twitter), I also initially though that how to [find all the IPs belongs to a particular](https://stackoverflow.com/questions/11164672/list-of-ip-space-used-by-facebook) domain and i might need to add them to routing table or something like that, But it's not required here.
    ok that is for your information about AS codes and IPs , now run the

    ```shell
        ipset -L twitter
    ```

    command and check whether the member IP have resolved
    Configuring the `mwan3` can be done via the UI too. But i will put the configuration elements here for convenient.
    I assume that you are somewhat familiar with OpenWRT and `mwan3` configurations, Hence will directly jump into mwan3 config.
    `mwan3` config is located in

    ```shell
    /etc/config/mwan3
    ```

    and following is a sample configuration

    ```
        config rule 'facebook'
            option proto 'all'
            option family 'ipv4'
            option sticky '1'
            option ipset 'facebook'
            option use_policy 'testp'
    ```

    In above rule config,
    option sticky: Is important to set true in UI or '1' in the config
    option ipset: Is the IP set name created with `ipset -N {name}` and the `ipset={domain}/{name}` in the config file
    option use_policy: is the policy name which have the WAN as a member that you want to forward the request match with the above ipset (i:e facebook.com requests go to Dialog 4G WAN)
    Change the above config accordingly and put it in the `mwan3` config at the top (will get the priority from configs in top to bottom)
    ![image](/images/posts/openwrt_mwan3/rules.png)
    Finally restart the `mwan3` service

    ```shell
    mwan3 restart
    ```

    and check the `mwan3` status

    ```shell
    mwan3 status
    ```

    or check the status from the web UI
    ![image](/images/posts/openwrt_mwan3/mwan3state.png)
    it should show the above added new rul in `Active ipv4 user rules:` section

# Verifying the changes

## Router

Login to the router via ssh and check the following outputs

-   IPset members

```shell
ipset -L {name}
```

run the above command and check whether any member IPs have resolved

-   mwan3 configs

```
mwan3 status
```

run above command and check whether the newly added rule is listed in the active ipv4 user rules section

-   Check dnsmasq configuration syntax using below command

```shell
dnsmasq --test
```

## External device (Laptop)

```shell
traceroute facebook.com
```

execute the above command and check from which WAN connection or hope the request been routed, Is it routing via the expected WAN connection

# Monitoring traffic

This is about monitoring the home network traffic using the OpenWRT, This is a different topic than the above discussed topic.

![image](/images/posts/openwrt_mwan3/statsgraph.png)

I have used [luci_app_statistics](https://openwrt.org/docs/guide-user/luci/luci_app_statistics) package in OpenWRT, It provide good coverage of parameters, from Network interfaces, to memory & CPU usages and there are lot more supported in `collectd`. This package gives more similar graph outputs this is available in [LERAN traffic graphs](https://lnms.learn.ac.lk/traffic/int.php). So i believe this `luci_app_statistics` package give more professional looking stats in OpenWRT routers.

The default `rrd` configuration store the monitoring data in the temporary directory (`/tmp/`) hence the graph data get cleared when router is restarted. In the above doc , they have pointed to some scripts and docs which explains how to backup the data to an external device plugged into router's USB interface. Only thing lacking there is how to use existing vFat/Fat32 formatted pen drive to backup the data. So I had to google and collect the information from various sources, and though of organizing them here.
To get [vFat/FAT32 partition](https://openwrt.org/docs/guide-user/storage/filesystems-and-partitions) mounted you need following package in the router

```shell
opkg install dosfstools
```

And then simply follow the instruction in [usb-installing](https://openwrt.org/docs/guide-user/storage/usb-installing) and [usb-drives](https://openwrt.org/docs/guide-user/storage/usb-drives)
To take a backup every X (i:e 6 hours) interval, You can use [this script](https://gist.github.com/squarewf/a2347fe44e217a19998eb2b6b1b16c59) This sheduling is done using [the corntabs](https://openwrt.org/docs/guide-user/base-system/cron)

# Schedule WiFi and switch WAN usages timely

To edit the `crontab` type

```shell
crontab -e
```

and following scheduled commands will switch off the wifi radios at mid night and switch on at 5AM morning.
And also it will switch the MWAN3 policies at mid night and restore back at 8AM morning(That's where peak time start for SLT connections)

```crontab
0 */6 * * * /etc/init.d/rrdbackup backup
0 8 * * * uci set mwan3.all.use_policy='Dialog4G_Only'; uci commit; mwan3 restart
0 0 * * * uci set mwan3.all.use_policy='SLT_Only'; uci commit; mwan3 restart
0 0 * * * /sbin/wifi down
0 5 * * * /sbin/wifi up
```

# References

-   This [Openwrt forum thread](https://forum.openwrt.org/t/mwan3-rules-with-ipset/52577/23) healped me a lot to figure out this path.
-   And this [StackOverflow answer](https://stackoverflow.com/questions/48592840/applying-ipset-in-lede) helps to find out about ipset list command and important of restarting and setting up the order for startup script (set ips before 19)
-   This [blog post in nicedoc.io](https://nicedoc.io/hardenedlinux/Debian-GNU-Linux-Profiles/blob/master/docs/dns/domain-name-based-routing.md) was also helpful.
-   I created this [github repo](https://github.com/tmkasun/openwrt_configs) with all the configuration files and script for reference


OpenWRT conditional hostname routing using MWAN3 and ipset


##### Note

> This article does **not** promote nor advice on any illegal, unlawful or unethical importation of any goods into Sri Lanka, If you are planing for such thing this is not the article for you !!
> The intention of this post is to help with the importation process of tech items to Sri Lanka thorough my experience on importing a [WiFi router](https://www.linksys.com/us/p/P-WRT1900ACS/) for my personal use.
> There are various combinations of items, specifications, quantity, purpose etc that leads to completely different experience/process for the importers. (i:e some gadgets (Drones :) ) requires Ministry of Defence approval , Ministry of Health approval and the list goes on and on). So I don't know every one of them and don't cover every thing in this article.

## Know your HS code

Harmonized System Codes or HS codes is simply an identification code for goods or items. HS codes is a standardize numerical method and widely used in my countries to categorize and identify goods in export ot import process. If not for such identification system, It's literally impressible to identify goods by their name.

![HS Code](/images/posts/importing_telecom/hscode.svg)

It's important to know the HS code of the good that you planing to import beforehand, So that you can estimate the tax, and know whether it requires any licence or even it's prohibited item.

You can find more information about HS codes from [here](https://www.trade.gov/harmonized-system-hs-codes)

To find the HS code for your item, You can just google search with the manufacturer name and the model name/number
i:e

```
Linksys WRT 1900 HS code
```

and you will probably find the HS code used in US exports. Then you can find the relevant taxes and tariff applies to the HS codes using the Sri Lanka customs website.
You can find the item description using the [HS finder](http://www.customs.gov.lk/business/hsfinder) and locate the relevant tariff information from [this page](http://www.customs.gov.lk/classification/tarrif)

## Obtaining a Telecommunications Regulatory Commission (TRC) licence

TRC will mainly look at the technical specification of the device, And validate whether it's conform to the Telecommunications regulations in the country. It has a wide spread scope of regulating various devices(RTT), Vendors and more, So what I have experienced is taking a licence for personal use WiFi devices. Obviously the procedure and fess will be differ from the licence type.

I have seen different set of required documents, that needs to be submitted for the licence.But mainly you need submit your NIC copy, and device technical specification, proforma invoice, Application form and etc included in a request letter(email).

Documents required to be submitted in order to issue Custom clearance letters ([source](http://www.trc.gov.lk/images/stories/spectrum/LPspectrumAllocantion1.pdf))

1. Request letter
2. [Application Form](http://www.trc.gov.lk/images/stories/spectrum/LowPowerApplication.pdf) ([Download](http://www.trc.gov.lk/low-power-device.html))
3. Copy of proforma/commercial invoice
4. Technical Specifications
5. Vendor’s Licence where applicable

For technical specification which has the frequency ranges, maximum gain value(s), RF Exposure Information, ect can be found in the product manual most of the time, If not you can search for the device's Federal Communications Commission ID (FCC ID) from the internet and provide those information along with the application.
For example , FCC ID of the Linksys WRT 1900AC V2 router is `Q87-WRT1900ACV2` and you can search for the FCC ID in [fccid.io](https://fccid.io/Q87-WRT1900ACV2) or [fcc.gov](https://www.fcc.gov/oet/ea/fccid)

## Import control licence

Thanks to the pandemic, Now you can apply for ICL via email, You have to fill the information and submit the application form to xxxxx email.

Download the [APPLICATION FOR AN IMPORT CONTROL LICENSE](https://srilankatradeportal.gov.lk/kcfinder/upload/files/DoIEC7AL.pdf)

> **_NOTE:_** I accidentally tries to access `https://srilankatradeportal.gov.lk/kcfinder/upload` and it redirected me to Bangladesh Trade Portal :D, seems like we have taken a clone of their portal and has forgot to change the 302 :D

For more information call `0112326774`,This is the general number of the [Department of Imports and Exports control](imexport.gov.lk) and ask for unit 4 or extension 119.

[This page](http://www.imexport.gov.lk/index.php/en/notices/112-instructions-to-the-applicants-who-personally-importing-phones.html) is also a good source to understand the process of obtaining a license for privately imported hand phones.

## Why all this hassle

![Narcotics](/images/posts/importing_telecom/Customs-seizes-narcotics-hidden-as-dried-leaves-Newsradio.png)

> Source : [News Radio](https://www.newsradio.lk/local/customs-seizes-narcotics-hidden-as-dried-leaves/)

All these regulatory bodies and authorities are there to protect the civilian's rights and ensure the well being of all the people in the society, So it's our responsibility to adhere to these guidelines and laws and support.
But at the same time there is a lot to improve on these process to make it smooth for actual buyers.

![Johnny-FD](/images/posts/importing_telecom/Johnny-FD-Posts-Facebook.png)

> Source : [Facebook](https://www.facebook.com/JohnnyFDK/posts/2902701656625133)

## Reference

-   [Drawing](https://excalidraw.com/#room=27703d66db1f97033a51,jKOLhWHyhhqd683XlJeo4Q)


From Google Photos ☁ to Immich 🖼️

Why I moved from cloud photo storage to Immich, and how I set up a local photo server at home.
3 min read

React State initializer function

2 min read

React fetch in a loop

Fetch data in a loop and render with data
4 min read

How React hooks work

Explanation of how react hooks works using JS Closures
1 min read

Sandpack with NextJS

Embed vscode into NextJS blogs
2 min read

How to customize WSO2 API Manager 3.x portals

Building and customizing APIM 3.x portal apps
11 min read

Visualizing FLIR Lepton Thermal data for statistical analysis

5 min read

OpenWRT conditional hostname routing using MWAN3 and ipset

This article will describe multiple OpenWRT configurations that i have done to load balancing two WAN connections, Collecting router statistics, and mounting USB pen/flash drive and persisting RRD data
8 min read

How to import telecommunication equipments to Sri Lanka

Legal process of importing telecommunication equipments such as mobile phones , WiFi routers, networking equipments to Sri Lanka
4 min read

From Google Photos ☁ to Immich 🖼️

React State initializer function

2 min read

React fetch in a loop

Fetch data in a loop and render with data4 min read

How React hooks work

Explanation of how react hooks works using JS Closures1 min read

Sandpack with NextJS

Embed vscode into NextJS blogs2 min read

How to customize WSO2 API Manager 3.x portals

Building and customizing APIM 3.x portal apps11 min read

Visualizing FLIR Lepton Thermal data for statistical analysis

5 min read

OpenWRT conditional hostname routing using MWAN3 and ipset

This article will describe multiple OpenWRT configurations that i have done to load balancing two WAN connections, Collecting router statistics, and mounting USB pen/flash drive and persisting RRD data 8 min read

How to import telecommunication equipments to Sri Lanka

Legal process of importing telecommunication equipments such as mobile phones , WiFi routers, networking equipments to Sri Lanka4 min read

Fetch data in a loop and render with data
4 min read

Explanation of how react hooks works using JS Closures
1 min read

Embed vscode into NextJS blogs
2 min read

Building and customizing APIM 3.x portal apps
11 min read

This article will describe multiple OpenWRT configurations that i have done to load balancing two WAN connections, Collecting router statistics, and mounting USB pen/flash drive and persisting RRD data
8 min read

Legal process of importing telecommunication equipments such as mobile phones , WiFi routers, networking equipments to Sri Lanka
4 min read