# Digital Twin
## Overview
You can test your operations and robot’s work in Aitheon Sim software, which is a digital playground embedded in Smart Infrastructure.
This allows you to make a prototype and test your infrastructure with Aitheon Core and simulated hardware and robots even in it prior to building a facility and actually buying robots for it.
Here's the general flow for setting up the simulation:
*Load a CAD designed robot model -> set up properties for its parts as for physical objects -> set up joints between immovable and movable parts -> set up sensors -> set up a scene for the simulation*.
In the following tutorial, we are using the Yezhik robot model by Aitheon as an example. Please, be aware that all the properties\` values here are for this particular model.
Aitheon robots simulation models are available in Marketplace, already configured for simulations.
## Steps to Make a Digital Twin
1. Import a robot model
2. Create a collision scene
3. Set up physics scene
4. Prepare robot to reflect physics
5. Set up robot\`s joint
6. Add sensors and measurements
7. Create and link camera
8. Debug
9. Add a robot to a scene
10. Give task and test
11. Avoid possible issues
## 1. Import a Robot Model
To load a CAD software designed robot model (STEP format):
1\) Go to menu *Window->Isaac->Step importer* and pick a **.stp** file.
Depending on the model detailing it may take a while.
2\) When a **Step import** window will appear, scroll it down and click **Finish Import** and choose a directory to save converted model objects.
3\) Go to menu *File->Save As* and save the model as a single USD - this is the 3-d model format that the simulator uses. After that, you can easily open this model by loading this file: menu *File->Open*.
The right **Stage** tab contains all the objects in the tree. The imported robot is the **Root** object in this instance.
4\) Change the view if needed: toggle **Perspective** to Top, Front, or **Right** view in **Viewport**. Drag the view with the right mouse button pressed and hit **F** after these moves to center the view back to the chosen object (part). The mouse wheel zooms the view in and out.
5\) Change the position if needed: switch to Rotate selection mode in Viewport and drag the sphere or go to the Details tab and change rotation and position numbers.
6\) Group minor parts into bigger containers. Choose parts holding Ctrl, right-click, and hit **Group Selected**. All immovable parts can be joined in the **chassis** group, for example. Also, you can drag-n-drop elements to a group.
7\) Also, for your convenience right-click and rename parts and groups of the object.
## 2. Create a Collision Scene
We need real physics in our simulation to test the robot. So we have to add a physics scene that imitates it.
1\) Go to menu *Physics->Add->Physics Scene*.
In the Stage tab, the **World** object will appear.
2\) In the same way add ground: *Physics->Add->Ground Plane*.
It's a basic plane for the robot\`s collision with the environment.
Note: ground plane appears at the Stage tab in the **Root** object (the robot model object name in our example) and is called **staticPlaneActor**. So when you manipulate the robot model, the ground plane will move with it. To unleash this plane and bind it to the **World** object - drag and drop **staticPlaneActor** from **Root** to **World** or the higher level (as **Root** and **World**).
### Adjust Position
Choose an object (root or **staticPlaneActor**) in the **Stage** tab to adjust its position and in the **Details** tab specify needed coordinates.
## 3. Set Up Physics Scene
We need to set up the physics scene in order to receive feedback from the simulator environment.
1. Expand the robot model object in the **Stage** tab (in our example it is called **Root**), find and click physicsScene.
2. Choose **PhysX Properties** lower-right tab.
3. Remove **enableGPUDynamics** flag.
4. Set **collisionSystem**: PCM, **solverType**: PGS, **broadphaseType**: MBP.
{% hint style="info" %}
Usually, default presets are ok, but these settings work better.
{% endhint %}
## 4. Prepare Robot to Reflect Physics
To reflect physics an object (robot model) should have **RigidBodies** properties. Apply these properties to every element (or group of elements) of the object, but not to the top wrapper (here called **Root**).
1\) Click on each part (a group of parts) that will interfere with the environment and add the property: *Physics->Set->Rigid Body*. In the **PhysX Properties** tab new properties - **Physic Body** and **PhysX Rigid Body** - will appear.
{% hint style="info" %}
Do it one by one with all the parts (or groups). It will not work for multi-selected parts (and groups) in most cases.
{% endhint %}
{% hint style="info" %}
If you first give all the parts **Rigid Body** properties and then group them, these parts will fall apart during the simulation, because the system treats them as separate. So it's better to group them first and then give **Rigid Bodies** to the whole group. Another solution: you can add **Fixed** joints to these separate parts (see the [**Joints**](https://app.gitbook.com/@aitheon/s/smart-infrastructure/~/drafts/-MPdPKjOgGS6Rpph3B0Z/main-tabs-in-smart-infrastructure-service/digital-twin#5-set-up-robot-s-joints) chapter).
{% endhint %}
2\) In the **PhysX Properties** tab of each element that you want to participate in collisions go to **Physics Prim Components** and **Add Prim Components**: **CollisionAPI** and **MassAPI:mass**.
**MassAPI:mass** will allow you to specify the **Mass Properties**. Otherwise, the defaults will be used (the part\`s geometry multiplied by a density of 1000).
In the chassis part of the robot, there are a few elements that interact with the environment. So you may want to delete the **CollisionAPI** from internal elements (this will make the simulation “lighter”):
3\) Choose the chassis group and go to *Physics->Remove->Collider*. This will remove all the **collision API**s.
4\) Select and apply **CollisionAPI** in **PhysX Properties** to external elements one by one.
## 5. Set Up Robot\`s Joints
Joints give you the ability to connect rigid bodies in ways that are not entirely rigid. A good example is a car: the wheels of the car revolve around their respective axes, the suspension can slide up and down along an axis. The technical term for the former is “revolute joint”, the second is a “prismatic joint”.
To add a joint to your scene, first select the two objects to connect. It is also possible to select a single joint in order to connect it to the world frame. Then select *Physics->Create->Joint*. In a submenu, you will be able to select the type of joint.
Articulations is an advanced, hierarchic mode of joints, useful for creating hierarchical mechanisms such as a vehicle or a robot. To use articulations, one should organize the joints of a mechanism and the objects they connect into a tree structure. For example, to create an articulated wheelbarrow, one would create the body (tray) object, which would have a child revolute joint for the wheel axis, and the joint would have a child wheel body. Articulated joint links parts starting from the articulation root to the last chained connection. The top tree wrapper should have **ArticulationAPI** in order to work correctly in the future.
The graph of joints connecting bodies will be parsed starting at this body, and the parts will be simulated relative to one another, which is more accurate than conventional jointed simulation.
There are several types of joints, mainly used are **RevoluteJoint** or just **PhysicsJoint** (it\`s a basic type of joint without additional API\`s ).
A description of joints and their behavior can be found [here](https://docs.nvidia.com/gameworks/content/gameworkslibrary/physx/guide/Manual/Joints.html).
{% hint style="info" %}
**PhysicsJoint** is not listed here because there is no such type in the menu. This type is basic and concerns every other type. Usually, we use this type for a root joint with **Articulation Joint** in **PhysX Properties**.
{% endhint %}
A description of **Articulated Joints** can be found [**here**](https://docs.omniverse.nvidia.com/app_create/prod_extensions/ext_physics.html).
### Step 1. Apply ArticulationAPI to top tree wrapper
Add a method of building the joints chain with the model.
1\) Select top tree wrapper (**Root**).
2\) In the **PhysX Properties** tab and add **ArticulationAPI**.
3\) Set **solverPositionIterationCount** in **PhysiX Articulation** properties to 64.
4\) Set **solverVelocityIterationCount** to 16.
{% hint style="info" %}
You can set up these two parameters to higher numbers for better precision, but it will load the system a lot.
{% endhint %}
### Step 2. Create ArticulatedRoot
Make a root object for all joints to connect to.
1\) Select top tree wrapper (**Root** in our example).
2\) Go to *Physics ->Add ->Joint ->To World Space*.
3\) Select the newly created joint in the **Stage** tab.
4\) Go to the **PhysX Properties** tab and **Remove Join Component** that is present there.
5\) **Add Joint Component** named **ArticulationJoint**.
6\) Scroll down to the **Physics Articulation Joint** property and change **articulationType** to **articulatedRoot**.
7\) Add a tab to the editor: go to *Window ->Isaac ->Relationship Editor*.
8\) A new **Relationship Editor** tab will appear, open **body0**, change the **0** path to your chassis object (for example, if you grouped all the chassis parts to **chassis** group in **Root**, the path will be **/Root/chassis**), and click **Modify**. Now the root object is attached to the chassis.
### Step 3. Create Joints for movable parts
Create all joints for all movable parts of the robot (model). If there is an immovable part that wasn\`t grouped with the rest immovable elements and got common **Rigid Body** properties you should create a joint for it too. Choose **Fixed** type in this case. If you don't do this the ungrouped and unjointed element will fall apart from the model during the simulation.
1\) Select two Rigid Bodies: the primary one first, then the secondary one. A joint will be created as a part of the second component (and will appear in the second component\`s submenu in the **Stage** tab).
2\) Make sure you have the correct joint type set in *Physics ->Joint Attributes->Type* - **Prismatic** (or maybe **Fixed**).
3\) Choose the connection type: *Physics->Add->Joint->Between Selected*.
4\) Select the new joint in **Stage** and in the **PhysX Properties** tab **Add Joint Component** - **ArticulationJoint** API.
5\) When you select the joint in **Stage** it gets visible on the **Viewport** tab. Move the joint to the correct place and apply the correct rotation (align its position and movement directions to actual elements of the model - use arrows dragging and **Rotate** mode sphere to move the joint). It doesn't have to be 100% precise, because the **ArticulationJoint** API will solve minor inaccuracies.
{% hint style="info" %}
When you move the joint element with the mouse button pressed and then release it, the selection highlighting will jump to another object. To switch back to the joint selection press **Ctrl+Z**. Or click the joint element on **Stage** again.
{% endhint %}
6\) In the joint\`s **PhysX Properties** tab scroll to **PhysX Joint** and put a flag to **enableCollision** property.
7\) If this is a joint for a driving wheel: add **Drive API** in **PhysX Joint Components** properties, scroll to **Joint Drive** properties, and set **angular:targetType** to **velocity**, **angular:type** to **acceleration**, and **angular:damping** to **10000**.
### Step 4. Repeat Step 3 for all the movable parts.
## 6. Add Sensors and Measurements
### Add Lidar
Lidar is a special Isaac Sim component for measuring distances (though laser beam reflection measurements). Lidar beams in the simulation will ignore anything that doesn’t have a collision API attached to it.
1\) In **Stage** (or in **Viewport**) select an object that represents the lidar, then click *Create->Isaac->Sensors->Lidar*.
{% hint style="info" %}
To hide an element choose it and press H. To unhide - press again or go to Edit->Unhide all.
{% endhint %}
2\) In the **Details** tab set the Z-axis **Position** on **1.3** to move the lidar up.
3\) In the **Other** section of **Details** enable **drawLidarLines** and **drawLidarPoints**. It's optional, but useful for debugging. For example, if you start a simulation and don't see “laser beams” of the lidar, you didn't set up **minRange** properly (see next).
4\) In **Others** set **maxRange** to **16**, **minRange** to **0.08**, **rotationRate** to **12**.
{% hint style="info" %}
These parameters are for the example model of the Yezhic robot. Use your values for your devices.
{% endhint %}
### Add IMU
An inertial measurement unit (IMU) is an electronic device that measures and reports a body's specific force, angular rate, and sometimes the orientation of the body.
In Isaac Sim, IMU can be represented as a simple cube shape with rigid body properties, but with disabled collision.
{% hint style="info" %}
If you already have IMU in your model, you can apply **Rigid Body** property to it and go to step 5 at once.
{% endhint %}
1\) Create a shape inside the root wrapper: right-click on Root (or another name you gave to the root object), then *Create->Shapes->Cube*.
2\) Select this **Cube** object and move it in the middle of the robot model in **Viewport**.
3\) Scale it to the usual IMU size in the **Details** tab by changing **Scale** numbers on the **X**, **Y**, and **Z** axes.
{% hint style="info" %}
You can slide the numbers in the axes fields - left and right by holding the left mouse button.
{% endhint %}
4\) Make it invisible: *Details tab->Other->purpose*: **guide**.
5\) Create a joint between the IMU and the chassis: select the IMU and the chassis group, *Physics->Add->Joint* (choose any type of it), in the **PhysX Properties** tab **Remove Joint Component** a present API and **Add Joint Component** - **ArticulationJoint**.
### Creating REB Components
The Robot Engine Bridge (REB) extension enables message communication between the two platforms (Isaac Sim with Isaac SDK™) over TCP to perform a robot simulation. These messages include simulated sensor data, drive commands, ground truth state of simulated assets, and scenario management.
Mainly used REB components are:
* Differential Base - for wheels moves simulation.
* Lidar - lidar simulation.
* RigidBodies Sync - objects interaction for multi-robots simulation.
#### Differential Base REB
1\) Select the root object and go to *Create->Isaac->Robot Engine->Differential Base*. This will create **REB\_DifferentialBase** in the **Root** object.
2\) In the **Relationship Editor** tab set **chassisPrim** path to the top wrapper (**/Root** in our instance).
3\) In **Details** set **leftWheelJointName** and **rightWheelJointName** accordingly (important: we type the wheels joints' names, but not wheels\` names!) and press **Enter** after each to save.
4\) In **Details** set the robot\`s direction vector **robotFront** to **1**, **-2**, **-1**.
5\) If there is a **Proportional** gain field, it should be set to **3**.
6\) Set **WheelBase** to **0.406**, **WheelRadius** to **0.1**.
{% hint style="info" %}
These parameters are for the example model of the Yezhic robot. Use your values for your devices. Also, you may want to change some default values, **maxSpeed**, for example.
{% endhint %}
#### Lidar REB
In Relationship Editor set the path for lidarPrim to the lidar component (created through *Create->Isaac->Sensor->Lidar*). Important: the lidar component in **Stage** must be **Lidar** type, not **Mesh** or else.
#### RigidBodies Sync REB
In **Relationship Editor** set path in **rigidBodyPrims** **0** to chassis, in **rigidBodyPrims** **1** to IMU.
## 7. Create and Link Camera
You can add extra points of view to your simulation by adding virtual cameras. For example, you can add a camera to the robot model and switch to its view.
1\) Choose the view for a new camera and create the camera through *Create->Camera* in the **Perspective** menu of **Viewpoint**.
2\) You can switch to the created **Camera** by picking it in the **Perspective** menu.
3\) If you want to change the camera\`s perspective, choose it, and move the point of view to the position from which you want to observe the real-time scene (holding the right mouse button).
4\) To bind the camera to the robot (so that the point of view will follow the robot moves): select the camera object in the **Stage** tab, right-click and create a new group (consist of one element - the camera). This is done because the **Rigid Body** property can be applied only to **Xform** elements (see in **Stage**) - groups.
Apply **Rigid Body** properties to the created group (*Physics->Set->Rigid Body*).
5\) Create a physics joint like for [**IMU**](https://app.gitbook.com/@aitheon/s/smart-infrastructure/~/drafts/-MPdY-4IxFj2FTLYSF3m/main-tabs-in-smart-infrastructure-service/digital-twin#add-imu).
## 8. Debug
In order to see collision shapes and debug this in real-time:
1. Go to *Physics->PhysX Debug Window*.
2. Move the tab to a convenient place for you.
3. In **Show** **collision shapes** pick **Selected**. If you choose **All** - the representation will become too “heavy”.
4. Collision shape movement is being shown only when you press the **Step** button, even if you run a scene from this window it won’t be constantly showing you collision form change.
## 9. Add a Robot to a Scene
Presumably, you want to test your virtual robot model in some virtual environment.
After you set up your robot virtual model and saved it in a **.usd**, create the scene, and add the robot to it:
Go to the **Content** tab, choose the model file, drag and drop it to the scene.
To connect the simulation to Smart Infrastructure go to the **Robot Engine Bridge** (**1**) tab and press **Create Application** (**2**). Then click '**play**' (**3**) to start the scene:
## 10. Give a Task and Test
Add the robot virtual model as a device, and a map of the virtual scene as a floor to the Smart infrastructure service (See [**Add Floor**](/smart-infrastructure/smart-infrastructure-service-quick-start/add-floor.md) and [**Add Device**](/smart-infrastructure/smart-infrastructure-service-quick-start/add-device.md)).
Launch an application to process the connection of the scene to Smart Infrastructure.
[**Create Task**](/smart-infrastructure/smart-infrastructure-service-quick-start/create-a-task.md) and watch the execution.
## 11. Possible Issues and Solutions
1\) If the root joint will have a relationship to the wrong body (which doesn’t contain anything to link) it will crash i.e. if you forget to change the relationship after creating this joint.
What to do: make sure you change the relationship to be connected with the chassis.
2\) Sometimes when you create/delete an element the system doesn’t correctly handle this and crashes when you start the scene.
What to do: save and reopen the scene.
3\) The system will crash when creating a joint between bodies that are not rigid.
What to do: make sure that all bodies that you’re creating joints with are rigid (have a **Rigid Body** property).
---
# Agent Instructions: Querying This Documentation
If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.
Perform an HTTP GET request on the current page URL with the `ask` query parameter:
```
GET https://docs.aitheon.com/smart-infrastructure/main-tabs-in-smart-infrastructure-service/digital-twin.md?ask=
```
The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.
Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.