Teaching WIKI
- teawiki.net -

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
~~NOTOC~~

====== Computer Vision and Robotics with NVIDIA Jetson  ======

The NDIVIA Jetson family consists of a range of embedded computers with a CPU as well as a GPU acceleration.

  * Jetson Nano 4GB
  * Jetson Xavier NX


===== Personal Playground Pages =====

  * **[[.:jetson:playground:Rolf:start]]**
  * **[[.:jetson:playground:Team:start]]**


===== Object Detector Implementations and Benchmarks =====

  * [[.:jetson:benchmarks:start]]
 
===== NVIDIA Jetson AI Fundamentals =====

^  [[https://developer.nvidia.com/embedded/learn/jetson-ai-certification-programs#course_outline|NVIDIA Jetson AI Fundamentals Course Outline]]                                                                  |^ [[https://www.youtube.com/playlist?list=PL5B692fm6--uQRRDTPsJDp4o0xbzkoyf8|YouTube Playist]]  ^ [[https://forums.developer.nvidia.com/c/agx-autonomous-machines/jetson-embedded-systems/70|Developer Forum]]  ^
|                                                                                                                                                                                                                                                                                                                                                                                                                              ||||
|  **S1**                                                                                                                                        |  **Getting Started with AI on Jetson Nano (by Dana Sheahan)**                                                                                                                                                                                                                |||
| S1E1                                                                                                                                           | First Time Setup with JetPack                                  | [[https://youtu.be/uvU8AXY1170|YouTube]]                                                      | Problem: \\ Videos and docker examples of S1 are based \\ on Jetpack 4.4.1 but not Jetpack 4.5.               |
| S1E2                                                                                                                                           | Hello Camera                                                   | [[https://youtu.be/zsjcSapzUfU|YouTube]]                                                      |                                                                                                               |
| S1E3                                                                                                                                           | tbd                                                            |                                                                                               |                                                                                                               |
| S1E4                                                                                                                                           | tbd                                                            |                                                                                               |                                                                                                               |
|                                                                                                                                                                                                                                                                                                                                                                                                                              ||||
|  **S2**                                                                                                                                        |  ** JetBot (by Jim Benson from JetsonHacks.com)**                                                                                                                                                                                                                            |||
| S2E1                                                                                                                                           | JetBot Intro and Hardware                                      | [[https://www.youtube.com/watch?v=9Wv9A6C6U5w|YouTube]]                                       | https://jetbot.org/master/                                                                                    |
| S2E2                                                                                                                                           | JetBot Software Setup                                          | [[https://youtu.be/MY_Fe7EN6ro|YouTube]]                                                      | https://github.com/NVIDIA-AI-IOT/jetbot                                                                       |
| S2E3                                                                                                                                           | JetBot Collision Avoidance                                     |                                                                                               |                                                                                                               |
| S2E4                                                                                                                                           | JetBot Road Following                                          |                                                                                               |                                                                                                               |
|                                                                                                                                                                                                                                                                                                                                                                                                                              ||||
|  **S3**                                                                                                                                        |  **Hello AI World (by Dustin Franklin aka Dusty-NV)**                                                                                                                                                                                                                        |||
| S3E1                                                                                                                                           | Hello AI World Setup                                           | [[https://youtu.be/QXIwdsyK7Rw|YouTube]]                                                      | https://github.com/dusty-nv/jetson-inference                                                                  |
| S3E2                                                                                                                                           | Image Classification Inference                                 | [[https://youtu.be/QXIwdsyK7Rw|YouTube]]                                                      | https://github.com/dusty-nv/jetson-inference                                                                  |
| S3E3                                                                                                                                           | Training Image Classification Models                           | [[https://youtu.be/sN6aT9TpltU|YouTube]]                                                      | https://github.com/dusty-nv/jetson-inference                                                                  |
| S3E4                                                                                                                                           | Object Detection Inference                                     | [[https://youtu.be/obt60r8ZeB0|YouTube]]                                                      | https://github.com/dusty-nv/jetson-inference                                                                  |
| S3E5                                                                                                                                           | Training Object Detection Models                               | [[https://youtu.be/2XMkPW_sIGg|YouTube]]                                                      | https://github.com/dusty-nv/jetson-inference                                                                  |
| S3E6                                                                                                                                           | Semantic Segmentation                                          | [[https://youtu.be/AQhkMLaB_fY|YouTube]]                                                      | https://github.com/dusty-nv/jetson-inference                                                                  |


===== More NDIVIA DLI Material =====

  * [[https://www.nvidia.com/en-us/training/|NVIDIA Deep Learning Institute]]
  * [[https://www.nvidia.com/en-us/training/online/|NVIDIA Self-Paced Online Training List]]
  * [[https://courses.nvidia.com/courses/course-v1:DLI+S-RX-02+V2/about|Getting Started with AI on Jetson Nano]]
  * [[https://github.com/dusty-nv/jetson-inference|Github: Jetson Inference by Dustin Franklin]] 
  * [[https://www.youtube.com/watch?v=L_JSLZQ2ZFs&feature=push-sd&attr_tag=quBuIvr68NJcx4wH%3A6|Accelerating Vision AI Applications Using NVIDIA Transfer Learning Toolkit and Pre-Trained Models]] (Youtube)

  * https://news.developer.nvidia.com/realtime-object-detection-in-10-lines-of-python-on-jetson-nano/

===== Some Material from JetsonHacks.com (Jim Benson) =====

  * [[https://github.com/JetsonHacksNano|JetsonHacksNano Gitub]]
===== NVIDIA GPU Cloud (MGC) Catalog Collections =====

  * https://ngc.nvidia.com/catalog/collections




===== Jetbot, HSRW Design =====

This 3D model was created by Harley Lara. Our goal is to create a model with does **not require digital fabrication**. Not many people have access to laser cutters or 3D printers.

| {{:workshops:nvidia_jetson:hsrw_jetbot_model_view_01.jpeg?direct&400|}} |  {{:workshops:nvidia_jetson:hsrw_jetbot_model_view_03.jpeg?direct&400|}} |
| {{ :workshops:nvidia_jetson:plywood_base_harley_v001.jpeg?direct&400|}} |

===== Interesting Projects =====

  * Face mask detection (VNC, YoloV4): \\ https://www.hackster.io/lrondan95/nvidia-jetson-nano-face-mask-yolov4-detector-ba816f
  * GTC 2016 – Flying Robot Demo: https://www.jetsonhacks.com/2016/04/11/gtc-2016-flying-robot-demo/

===== Our NVIDIA Jetson Nano Reference Setup =====

Status on 2021-01-28

^ Main Components                                                                                                                                 ^ Source                                                                                                                                                                                                                                        ^ Datasheet / Web Site                                                        ^
| NVIDIA Jetson Nano 4GB Developer Kit (B01)                                                                                                                             | [[https://www.welectron.com/NVIDIA-Jetson-Nano-4GB-Developer-Kit-B01|NVIDIA Jetson Nano 4GB Developer Kit (B01)]]                                                                                                                             | [[https://developer.nvidia.com/EMBEDDED/jetson-nano-developer-kit|NVIDIA]]  |
| Wifi Adapter Card                                                                                                                               | [[https://www.welectron.com/Waveshare-16578-Wireless-AC8265|Waveshare 16578 Wireless-AC8265]]                                                                                                                                                 |                                                                             |
| MicroSD                                                                                                                                         | [[https://www.welectron.com/SanDisk-Ultra-microSD-Speicherkarte-64-GB|SanDisk Ultra microSD Speicherkarte 64 GB]]                                                                                                                             |                                                                             |
| Rugged Metal Case                                                                                                                               | [[https://www.welectron.com/Waveshare-17855-Jetson-Nano-Case-C|Waveshare 17855 Jetson Nano Case (C)]]                                                                                                                                         |                                                                             |
| CPU Fan                                                                                                                                         | [[https://www.welectron.com/Waveshare-16681-Fan-4020-PWM-5V|Waveshare 16681 Fan-4020-PWM-5V]]                                                                                                                                                 |                                                                             |
| Stereo Camera                                                                                                                                   | [[https://www.welectron.com/Waveshare-17742-IMX219-83-Stereo-Camera|Waveshare 17742 IMX219-83 Stereo Camera]]                                                                                                                                 |                                                                             |
| Power Plug, 4A                                                                                                                                  | [[https://www.welectron.com/Meanwell-GST25E05-P1J-Steckernetzteil-5V-4A|Meanwell GST25E05-P1J Steckernetzteil 5V/4A]]                                                                                                                         |                                                                             |
^  Photos                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |||
|  {{ workshops:nvidia_jetson:img_20210201_125818.jpg?600&direct }} \\ NVIDIA Jetson Nano 4GB (B01) with Waveshare 17742 IMX219-83 Stereo Camera and Waveshare 17855 metal case.                                                                                                                                                                                                                                                                                                                                   |||

===== Our NVIDIA Jetson Xavier NX Reference Setup =====

Status on 2021-02-11

^ Main Components                                                                                                                                                     ^ Source                                                                                                                      ^ Datasheet / Web Site                                                                                                                                                         ^
| NVIDIA Jetson Xavier NX Developer Kit                                                                                                                               | [[https://www.welectron.com/NVIDIA-Jetson-Xavier-NX-Developer-Kit|NVIDIA Jetson Xavier NX Developer Kit]]                   | [[https://developer.nvidia.com/embedded/jetson-xavier-nx-devkit|NVIDIA]]                                                                                                     |
| Wifi Adapter Card                                                                                                                                                   | included in the kit: **AzureWave CB375NF** \\ I-PEX MHF4 Connector Receptacle (20449)                                       | {{http://www.azurewave.com/img/nvidia/AW-CB375NF_DS_B2_A_STD.pdf|AzureWave}}  \\ [[http://www.wellshow.com/rf-connectors/mini-connector/ipex-mhf-connector/|Conn. Compat.]]  |
| :!: 2 x Ext. Antennas, I-PEX MHF4 -> RP-SMA                                                                                                                         |   |                                                                                                                                                                              |
| MicroSD                                                                                                                                                             | [[https://www.welectron.com/SanDisk-Ultra-microSD-Speicherkarte-64-GB|SanDisk Ultra microSD Speicherkarte 64 GB]]           |                                                                                                                                                                              |
| Rugged Metal Case                                                                                                                                                   | [[https://www.welectron.com/Waveshare-17855-Jetson-Nano-Case-C|Waveshare 17855 Jetson Nano Case (C)]]                       |                                                                                                                                                                              |
| CPU Fan                                                                                                                                                             | included in the kit                                                                                                         |                                                                                                                                                                              |
| Stereo Camera                                                                                                                                                       | [[https://www.welectron.com/Waveshare-17742-IMX219-83-Stereo-Camera|Waveshare 17742 IMX219-83 Stereo Camera]]               |                                                                                                                                                                              |
| Power Plug, 4A                                                                                                                                                      | included in the kit                                                                                                         |                                                                                                                                                                              |
^  Photos                                                                                                                                                                                                                                                                                                                                                                                                                                                                        |||
|  {{:workshops:nvidia_jetson:xavier_img_02.jpg?600&nolink}} \\ NVIDIA Jetson Xavier NX with Waveshare 17742 IMX219-83 Stereo Camera and Waveshare 17855 metal case.                                                                                                                                                                                                                                                                                                             |||


==== Assembly of Xavier NX Dev Kit and Waveshare 17855 Jetson Nano Case (C) ====

The rugged metal case by Waveshare is well suited for the Xavier NX dev kit. A challenge are the Wifi antennas. A Wifi adapter card is already pre-mounted to the Xavier NX dev kit. The Wifi card is a AzureWave CB375NF (status of Feb. 2021). It is located at the bottom side of the Xavier NX carrier board. Two narrow PCB antennas (Wifi + Bluetooth) are connected via short cables providing Hirose U.FL (or similar I-PEX MHF) connectors and glued to the frame on the bottom side. 

The RF signal from these antennas would strongly attenuated by the metal case. It is advisable to connect external antennas.

In the part list above we list one example product consisting of two rod antennas mounted on SMA receptables perfectly fitting the dedicated antenna holes in the metal case.

A step-by-step description on how to mount the Jetson Xavier NX to the case and just rearranging the default internal PCB antennas is given here: **[[workshops:nvidia_jetson:xavier_nx:case_assembly|Jetson Xavier NX case assembly (without external antennas)]]**. **Thank you, Harley!** 

Another description how to mount the SMA antennas is here **[[workshops:nvidia_jetson:xavier_nx:case_assembly_antennas|Jetson Xavier NX case assembly (with external antennas)]]**.


===== Additional Components =====

These components can be used with both development kits, Nano and Xavier NX.

^ Additional Components                                                                                                                           ^ Source                                                                                                                                                                                                                                        ^ Datasheet / Web Site                                                        ^
| Webcam CS Mount \\ Sony IMX179 (8 MP)                                                                                                           | [[https://www.amazon.de/Variable-Compatible-Computer-Conferences-USB8MP02G-MFV/dp/B08BFYYSZ1/|USB8MP02G-MFV (5-50) ]] (various resellers, ELP, SVpro, etc.)\\ 8MP, Manual Varifocal Lens, CS Mount, Sony IMX179 Sensor, **MFV Case**                                    | [[http://www.elpcctv.com/-p-251.html|ELP]]                                  |
| Webcam CS Mount \\ Sony IMX179 (8 MP)                                                                                                           | [[https://www.amazon.de/Svpro-varioobjektiv-8megapixel-industriekamera-SV-USB8MP02G-SFV/dp/B07HJTMPRW/|USB8MP02G-SFV (5-50)]] (various resellers, ELP, SVpro, etc.)\\ 8MP, Manual Varifocal Lens, CS Mount, Sony IMX179 Sensor, **SFV Case**                          | [[http://www.elpcctv.com/-p-249.html|ELP]]                                  |
| USB-to-DC Power Cable                                                                                                                           | [[https://www.reichelt.de/de/en/converter-usb-a-connector-to-2-1-mm-dc-port-0-5-m-goobay-55157-p286917.html|GOOBAY 55157 Converter]] \\ USB-A connector to 2.1-mm DC port, 0.5 m \\ (J25 power jack, 5.5/2.1 mm, 9.5 mm deep, pos. polarity)  |                                                                             |
| Power Bank                                                                                                                                      | [[https://www.amazon.de/gp/product/B07YPSZ566/ref=ppx_yo_dt_b_asin_image_o03_s01|Iniu Power Bank, 20000mAh]]                                                                                                                                  |                                                                             |
^  Photos                                                                                                                                                                                                                                                                                                                                                                                                                                                                     |||
|  {{ workshops:nvidia_jetson:img_20210201_131538.jpg?600&direct }} \\ ELP Webcam USB 8MP 02G-SFV (5-50), Sony IMX179, Manual Varifocal Lens, CS Mount                                                                                                                                                                                                                                                                                                                            |||

===== NVIDIA Jetson Nano Developer Kit, rev. B01 (4GB, Dual Camera) =====

:!: **Even if you are using rev. B01 read the [[https://developer.nvidia.com/embedded/learn/jetson-nano-2gb-devkit-user-guide|Jetson Nano 2GB Developer Kit User Guide]]** (with carrier board rev. A02), too! Most information is still valid, e.g. how to use the FFC connectors for the camera cables. The GPIO pin header definition has not changed neither. 


**Sources**

  * [[https://developer.nvidia.com/embedded/downloads|Jetson Download Center]]
  * [[https://developer.nvidia.com/embedded/dlc/Jetson_Nano_Developer_Kit_User_Guide|Jetson Nano Developer Kit User Guide]] (pdf)
  * :!: [[https://developer.nvidia.com/embedded/learn/jetson-nano-2gb-devkit-user-guide|Jetson Nano 2GB Developer Kit User Guide]] (Website, old revision A02, still mostly valid)
  * ArduCam: [[https://www.arducam.com/nvidia-jetson-nano-b01-update-dual-camera/|Jetson Nano B01 vs A02: What’s New for the Compute on Module (CoM) and Carrier Board]]
  * https://www.waveshare.com/product/ai/boards-kits/jetson-nano/jetson-nano-developer-kit-b01.htm
  * [[https://www.jetsonhacks.com/nvidia-jetson-nano-j41-header-pinout/|JetsonHacks.com]]


Most important differences to the A02 version: 
  * The Carrier Board version B01 now provides two MIPI CSI-2 camera interfaces for FFC (FFC: Flexible Flat Cable). 
  * The barrel connector power jumper is located elsewhere.
  * The compute modules on A02 and B01 are not compatible. You cannot mix modules and carrier board versions. 

**:!: IMPORTANT**: Close the **JUMPER J48** if you want to use the **barrel connector** for power!


{{workshops:nvidia_jetson:nv_jn_devkit_carrier_board_b01.png?direct|}} \\
//Fig.: Developer kit module and carrier board: rev B01 top view.// 
\\ Source: [[https://developer.nvidia.com/embedded/dlc/Jetson_Nano_Developer_Kit_User_Guide|Jetson Nano Developer Kit User Guide]] 



==== Jetson Nano 40 Pin Expansion Header Pinout  (J41 on rev. B01)====

{{https://developer.nvidia.com/sites/default/files/akamai/embedded/images/docs/jetson/646984561.png?direct&600}} \\
//Fig.: 40-Pin Header // \\
Source: [[https://developer.nvidia.com/embedded/learn/jetson-nano-2gb-devkit-user-guide|Jetson Nano 2GB Developer Kit User Guide]].


[[https://www.jetsonhacks.com/nvidia-jetson-nano-j41-header-pinout/|JetsonHacks.com]] provides even a **[[https://www.jetsonhacks.com/wp-content/uploads/2019/05/Jetson-Nano-GPIO-mapping.xlsx|spreadsheet for the Jetson Nano pinout]]** for download.


==== MicroSD Card Insertion and Removal ==== 

{{https://developer.nvidia.com/sites/default/files/akamai/embedded/images/docs/jetson/631259101.png?150}} \\
//Fig.: MicroSD card insertion and removal. \\ Caution: The card is snapping into place! Do not pull is out with force!//

==== FFC Connector Handling (Flex Flat Cables for Cameras) ====

{{https://developer.nvidia.com/sites/default/files/akamai/embedded/images/docs/jetson/633427294.png?300}}
{{https://developer.nvidia.com/sites/default/files/akamai/embedded/images/docs/jetson/633427296.png?250}}

Source: [[https://developer.nvidia.com/embedded/learn/jetson-nano-2gb-devkit-user-guide|Jetson Nano 2GB Developer Kit User Guide]]

==== Case: Waveshare Jetson Nano Case Version C ====

  * https://www.waveshare.com/wiki/Jetson_Nano_Case_(C)
  * 


==== Boot Flow =====

{{https://docs.nvidia.com/jetson/l4t/Tegra%20Linux%20Driver%20Package%20Development%20Guide/images/image2_4.png?800&direct}}

Source: [[https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/bootflow_jetson_nano.html|NVIDIA Jetson Nano Boot Flow]]




===== Remote Desktop Sharing =====

https://www.tecmint.com/best-remote-linux-desktop-sharing-software/

  * [[.:jetson:remote desktop]]


===== ArduCam Jetson Compatible Cameras =====

  * [[https://www.arducam.com/sony/imx477/|Sony IMX477 Chip]]