New version (#2)

* Initial commit * Initial commit First Work version * Update README * reduce warning * fixed cmake windows install * depth_viewer.py 文件重命名 * fixed name * fixed name * add ESC to exit and fixed depth work mode * fixed readme * fixed unit test * fixed unit test * fixed format * add docs/Chinese docs/Orbbec SDK（Python）使用手册/Orbbec SDK（Python）使用手册.md * rename docs * add windows x64 OrbbecSDK lib * add docstring * update readme * add Readme_EN.md * update README * update README * update README * update README * update README * update sdk to v1.6.1 * add win x86 * fixed playback * fixed create tarball * chang line spartor by LF * update sdk to v1.6.1 * remove no utf8 code * fixed README * remove create tarball * fixed depth render * fixed depth render * update README * update readme * install udev rules scripts * update readme * update readme * update readme * update readme * remove extra blank * update readme * add check bad frame * fix frame to bgr image * fix frame to bgr image * fix frame to bgr image * update readme * fixed image covert * fixed install pybind11 * add help covert * add gemini config * update SDK * add package.xml * fixed ir mjpeg render * fix ir render * fix crash * fix crash * fix stop depth viewer * fix stop ir viewer * fix stop device * add depth align sample args * add print log * catch exception * update SDK to v1.6.2 * bump to v1.2.2 * remove unused code * fix save image to disk * fix save point cloud to disk * fix stop imu * fix add py::call_guard<py::gil_scoped_release>() * add max ir limit * fixed depth * add disable PYBIND11_NO_ASSERT_GIL_HELD_INCREF_DECREF * update README * check empty * change cmake default to release * remove unused code * update README * fix config file * add deeyea config * remove max ir limit * fix ir render * update SDK config * update SDK to v1.6.3 * bump to v1.2.4 * add device reboot API * update README * remove unused file * add print camera params in test * fix return string ABI compatibility * fix warning * fixed issue #1 * update SDK to v1.7.1 * update SDK to v1.7.1 * add double IR sample * update README * bump to v1.2.5 * add stubs * fixed get default left && right ir profile * update readme * update SDK to v1.7.2 * bump to v1.2.6 * remove ssize_t * add license header * fixed file name typo * add play back color frame * update readme * update SDK to sdk v.1.7.4 * add read baseline interface * update playback sample * bump to v1.2.7 * add filter for depth viewer * update SDK to sdk v.1.8.1 * add net work device sample * add net work device sample * remove h264 example for windows * update 1.8.1 header and lib file * update examples to support Femto Bolt. * Update the Readme document and add code comments * Modify the sample's readme document and add an introduction to net_device . * update .rules file to support more devices * update version to 1.2.9 * fixed point cloud scale * roll back * fixed compile error * add multi device config * add numpy format point cloud * add device sync sample * update readme * remove point cloud filter z > 0 * fixed crash * prepare release v1.3.0 * fixed macos compile error * fixed sdk replace error * add setup.py * Add __repr__ method to OBCameraParam class * fixed print stream profile * add save point cloud by open3d * add get temperature interface * add get temperature interface test * update sdk to v1.9.5 * bump to v1.3.1 * add helper funtion bind * add OBCalibrationParam binding * fixed compile error * Add IMU data analysis and saving scripts * Add license headers to example files * Update data analysis scripts and add plot output folder to gitignore * update .gitignore * Update pybind11 version to 2.11.0 and add pybind11-global 2.11.0 to requirements.txt * Update CmakeList CMAKE_INSTALL_PREFIX * Gemini 330 series supoort (octo-models#36) * wip: add ton of new interface * Add more interface * Add more example * Update PointCloud filter * get post filter example * add hdr filter example * add set depth unit example * use plyfile save points simplfiy code * Update README * Update stubs * Update README * Update SDK to v1.10.5 * Fixed compile error * bump to v1.3.2 * fixed setup.py * fixed setup.py, add more note * add scripts/build_wheels.sh, not working now * Remove SDK stage version print statement * Update README * addbind OBGyroFullScaleRange type * add save_imu.py * ignore csv * update save timestamp scripts * Remove unused file * add multi device save data * add ob format new enum bind * fixed example crash * Check is pipeline start avoid crash * Fixed set depth work mode * Add timestamp unit * Format code * Update README for get windows uvc time stamp * remove get stage version * Update SDK to v1.10.8 * Add github action * use manual trigger action * Add enum net device interface * Update SDK config * Limit numpy<2.0 to fix issue 47 (octo-models#48) * Update: Bump to v1.3.3 * install new octo and debug the virtual env * Update SDK lib to version 1.10.11 * Update properties.cpp doc * add enable frame sync * test fine-tuned octo1.5 * add new env for ur5 to eval our model * optimize camera * caculate the Y matrix, code tag tracker * optimise code and Automated calibration process * slove the Matrix Mask * update sdk to v.1.10.12 * add gripper controller, camera, tag_tacker * Revert "add gripper controller, camera, tag_tacker" This reverts commit 7f9217a. * Revert "merge pyorbbecsdk" This reverts commit a7908a5, reversing changes made to 7f9217a. * Revert "merge pyorbbecsdk" This reverts commit a7908a5, reversing changes made to 7f9217a. * initial octo * add fun automatic_calibration, pybullet_ur5, camera, tag_tracker --------- Co-authored-by: Joe Dong <[email protected]> Co-authored-by: zhonghong <[email protected]> Co-authored-by: Lukas Koestler <[email protected]>
tongvibe · Aug 30, 2024 · d846793 · d846793
1 parent 5eaa5c6
commit d846793
Show file tree

Hide file tree

Showing 140 changed files with 113,967 additions and 21 deletions.
diff --git a/Log/OrbbecSDK.log.txt b/Log/OrbbecSDK.log.txt
diff --git a/examples/01_inference_pretrained.ipynb b/examples/01_inference_pretrained.ipynb
diff --git a/examples/04_eval_finetuned_on_ur5.py b/examples/04_eval_finetuned_on_ur5.py
@@ -0,0 +1,240 @@
+"""
+This script shows how to evaluate a finetuned model on a real UR5 robot.
+How to use it:
+python examples/04_eval_finetuned_on_ur5.py --checkpoint_weights_path=/home/tong/model_space/7.24/model_8_octo1.5_win2_h4_10k/ --checkpoint_step=9999 --im_size=256
+"""
+
+from datetime import datetime
+from functools import partial
+import os
+import time
+
+from absl import app, flags, logging
+import click
+import cv2
+import imageio
+import jax
+import jax.numpy as jnp
+import numpy as np
+import urx
+import gym
+from octo.model.octo_model import OctoModel
+from octo.utils.gym_wrappers import HistoryWrapper, TemporalEnsembleWrapper
+from octo.utils.train_callbacks import supply_rng
+
+np.set_printoptions(suppress=True)
+logging.set_verbosity(logging.ERROR)
+
+FLAGS = flags.FLAGS
+
+flags.DEFINE_string("checkpoint_weights_path", None, "Path to checkpoint", required=True)
+flags.DEFINE_integer("checkpoint_step", None, "Checkpoint step", required=True)
+
+# Custom to UR5 robot
+flags.DEFINE_string("ip", "192.168.51.254", "IP address of the robot")
+flags.DEFINE_float("acc", 0.5, "Acceleration for robot movements")
+flags.DEFINE_float("vel", 0.5, "Velocity for robot movements")
+flags.DEFINE_spaceseplist("goal_pose", [0.5, 0.0, 0.3], "Goal position and orientation")
+flags.DEFINE_spaceseplist("initial_pose", [0.5, 0.0, 0.3], "Initial position and orientation")
+flags.DEFINE_bool("blocking", False, "Use the blocking controller")
+flags.DEFINE_integer("im_size", None, "Image size", required=True)
+flags.DEFINE_string("video_save_path", None, "Path to save video")
+flags.DEFINE_integer("num_timesteps", 50, "Number of timesteps")
+flags.DEFINE_integer("window_size", 2, "Observation history length")
+flags.DEFINE_integer("action_horizon", 4, "Length of action sequence to execute/ensemble")
+flags.DEFINE_bool("show_image", False, "Show image")
+
+STEP_DURATION = 0.2
+
+
+
+def null_obs(img_size):
+    return {
+        "image_primary": np.zeros((img_size, img_size, 3), dtype=np.uint8),
+        "proprio": np.zeros((6,), dtype=np.float64)
+    }
+
+def convert_obs(pose, image, im_size):
+    image_obs = (image.reshape(im_size, im_size, 3)).astype(np.uint8)
+    # cv2.imshow('imagw',image_obs)
+    proprio = pose
+    return {
+        "image_primary": image_obs,
+        # "proprio": proprio,
+    }
+
+class UR5Env(gym.Env):
+    def __init__(self, ip, camera_ID=0, im_size: int=256, acc=0.5, vel=0.5):
+        self.rob = urx.Robot(ip)
+        self.cap = cv2.VideoCapture(camera_ID) 
+        self.cap.set(cv2.CAP_PROP_BUFFERSIZE, 1)
+        self.rob.set_tcp((0, 0, 0, 0, 0, 0))
+        self.rob.set_payload(0.1, (0, 0, 0.1))
+        self.acc = acc
+        self.vel = vel
+        self.im_size = im_size
+        self.observation_space = gym.spaces.Dict(
+            {
+                "image_primary": gym.spaces.Box(
+                    low=np.zeros((int(im_size), int(im_size), 3)),
+                    high=255 * np.ones((int(im_size), int(im_size), 3)),
+                    dtype=np.uint8,
+                ),
+                "proprio": gym.spaces.Box(
+                    low=np.ones((6,)) * -1, high=np.ones((6,)), dtype=np.float64
+                ),
+            }
+        )
+        self.action_space = gym.spaces.Box(
+            low=np.zeros((6,)), high=np.ones((6,)), dtype=np.float64
+        )
+
+    def step(self, action):
+        self.rob.movel((action[0], action[1], action[2], 0, 0, 0), acc=self.acc, vel=self.vel, relative=True)
+        obs = self.get_observation()
+        reward = self.compute_reward(obs)
+        done = self.check_done(obs)
+        truncated = False
+        return obs, reward, done, truncated, {}
+
+    def reset(self, seed=None, options=None):
+        super().reset(seed=seed)
+        initial_pose = [-0.1554021 + 3.1415926 / 2, -1.6309904, 2.4249759, -2.406821, -1.5576328, 2.7911022]
+        self.rob.movej(initial_pose, acc=self.acc, vel=self.vel)
+        return self.get_observation(), {}
+
+    def get_observation(self):
+        pose = self.rob.get_pose().pos
+        image = self.get_image()
+        return convert_obs(pose, image, self.im_size)
+
+    def compute_reward(self, obs):
+        # Implement your reward function
+        return 0
+
+    def check_done(self, obs):
+        # Implement your logic to check if the task is done
+        return False
+
+    def get_image(self):
+        # Implement image capture logic, e.g., from a camera
+        # Return a dummy image for now
+        ret, frame = self.cap.read()  # 从摄像头获取一帧图像
+        if ret:  # 如果成功获取到图像
+            frame = cv2.resize(frame, (self.im_size, self.im_size))  # 调整图像尺寸为 im_size x im_size
+            # cv2.imshow('imagw',frame)
+            return frame
+        else:  # 如果未能成功获取图像
+            return None  # 或者采取其他处理措施
+        # return np.zeros((self.im_size, self.im_size, 3), dtype=np.uint8)
+
+    def close(self):
+        self.cap.release()
+        self.rob.close()
+
+
+def main(_):
+    env = UR5Env(ip = FLAGS.ip, camera_ID=2, acc = FLAGS.acc, vel = FLAGS.vel)
+    env = HistoryWrapper(env, FLAGS.window_size)
+    env = TemporalEnsembleWrapper(env, FLAGS.action_horizon)
+
+    model = OctoModel.load_pretrained(
+        FLAGS.checkpoint_weights_path,
+        FLAGS.checkpoint_step,
+    )
+
+    def sample_actions(pretrained_model:OctoModel, observations, tasks, rng):
+        observations = jax.tree_map(lambda x: x[None], observations)
+        actions = pretrained_model.sample_actions(
+            observations,
+            tasks,
+            rng=rng,
+            unnormalization_statistics=pretrained_model.dataset_statistics["action"],
+        )
+        return actions[0]
+
+    policy_fn = supply_rng(
+        partial(
+            sample_actions,
+            model,
+            # argmax=FLAGS.deterministic,
+            # temperature=FLAGS.temperature,
+            # argmax=False,
+            # temperature=1.0,
+        )
+    )
+
+    goal_image = jnp.zeros((FLAGS.im_size, FLAGS.im_size, 3), dtype=np.uint8)
+    goal_instruction = ""
+
+    while True:
+        modality = click.prompt("Language or goal image?", type=click.Choice(["l", "g"]))
+
+        if modality == "g":
+            if click.confirm("Take a new goal?", default=True):
+                assert isinstance(FLAGS.goal_pose, list)
+                _pose = [float(e) for e in FLAGS.goal_pose]
+                env.rob.movej(_pose, acc=FLAGS.acc, vel=FLAGS.vel)
+
+                input("Press [Enter] when ready for taking the goal image.")
+                obs = env.get_observation()
+                goal = jax.tree_map(lambda x: x[None], obs)
+
+            task = model.create_tasks(goals=goal)
+            goal_image = goal["image_primary"][0]
+            goal_instruction = ""
+
+        elif modality == "l":
+            print("Current instruction:", goal_instruction)
+            if click.confirm("Take a new instruction?", default=True):
+                text = input("Instruction?Please:")
+            task = model.create_tasks(texts=[text])
+            goal_instruction = text
+            goal_image = jnp.zeros_like(goal_image)
+        else:
+            raise NotImplementedError()
+
+        input("Press [Enter] to start.")
+
+        obs, _ = env.reset()
+        time.sleep(2.0)
+        last_tstep = time.time()
+        images = []
+        goals = []
+        t = 0
+        while t < FLAGS.num_timesteps:
+            if time.time() > last_tstep + STEP_DURATION:
+                last_tstep = time.time()
+                # print(obs["image_primary"].shape)
+                images.append(obs["image_primary"][-1])
+                goals.append(goal_image)
+
+                if FLAGS.show_image:
+                    bgr_img = cv2.cvtColor(obs["image_primary"][-1], cv2.COLOR_RGB2BGR)
+                    cv2.imshow("img_view", obs["image_primary"][-1])
+                    cv2.waitKey(20)
+
+                forward_pass_time = time.time()
+                action = np.array(policy_fn(obs, task), dtype=np.float64)
+                print("forward pass time:", time.time() - forward_pass_time)
+
+                start_time = time.time()
+                obs, _, _, truncated, _ = env.step(action)
+                print("step time:", time.time() - start_time)
+
+                t += 1
+                if truncated:
+                    break
+        if FLAGS.video_save_path is not None:
+            os.makedirs(FLAGS.video_save_path, exist_ok=True)
+            curr_time = datetime.now().strftime("%Y-%m-%d_%H-%M-%S")
+            save_path = os.path.join(
+                FLAGS.video_save_path,
+                f"{curr_time}.mp4",
+            )
+            video = np.concatenate([np.stack(goals), np.stack(images)], axis=1)
+            imageio.mimsave(save_path, video, fps=1.0 / STEP_DURATION * 3)
+
+
+if __name__ == "__main__":
+    app.run(main)
diff --git a/pose_data.txt b/pose_data.txt
@@ -0,0 +1,8 @@
+{"base_to_hand": [[0.06457157997891422, -0.03168044965936879, -0.9974100762316402, -0.3351132425087788], [0.5090549351661208, 0.860715702421678, 0.005617169017691758, -0.2623014752534206], [0.858308559925883, -0.5080992311686143, 0.07170486206541227, 0.2967943862145207], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.9662527914124996, -0.2572601324439962, 0.013144099149424292, 0.14962684389098524], [0.24909945121306437, 0.94616272110202, 0.2067016415081696, -0.06432747109656006], [-0.06561254828843717, -0.196451850212003, 0.9783157282060727, 0.4859539089137055], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.31405473043883353, -0.06962810936118581, -0.9468482205062113, -0.2938012902375974], [0.9088055249070662, 0.2664782404131411, -0.32103249879834345, -0.4331163765646691], [0.2746673336736347, -0.9613226689166876, -0.020410341484484628, 0.21127238406603516], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.9292388430201153, -0.21858100710117045, -0.2978884287065816, 0.0911493110498086], [0.0406898707810201, 0.861874244983647, -0.5054870129387019, -0.17634288682550248], [0.36723222494555396, 0.4575971253936216, 0.8097810591715716, 0.6837250635198755], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[0.15079309322055953, -0.5219880949019406, -0.8395176423504263, -0.2867437092448589], [0.95744184553663, 0.28853071217804677, -0.007425668013117313, -0.10209277991710639], [0.24610273353294465, -0.8026695814036258, 0.5432816835095222, 0.36899037059750345], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.9565873322644194, 0.23362725342265778, 0.17423829145577158, 0.060673670169363245], [-0.09899457072450041, 0.8227585120078699, -0.5597039448544813, -0.0312963267809726], [-0.27411813277912334, 0.5181570585997854, 0.8101681997615714, 0.2277796407056609], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.8145458920698787, -0.19099273171309358, -0.5477561192217353, -0.07207119559726946], [0.5170590341410358, 0.18904224768107483, -0.8348131430474884, -0.29441097358115376], [0.262992290619373, -0.9632158662649084, -0.05522906888874594, 0.2990021930737376], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.5889237936747496, 0.020607848402062673, -0.8079257898024909, -0.21072644697186926], [-0.37477370705719604, 0.8926570322905043, -0.25041583656173994, -0.19754270771625804], [0.7160401062387134, 0.45026518773556834, 0.5334302456470811, 0.5773436095277129], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[0.06468399716774395, -0.03180327962297589, -0.9973988830531271, -0.33507726852408537], [0.5091400051826669, 0.8606656531033955, 0.00557572333413782, -0.26229054141837455], [0.8582496347992215, -0.5081763325592089, 0.0718636166197823, 0.29687019778934276], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.968559850979207, -0.24847172036490434, 0.012394323299041075, 0.14932225167182192], [0.2410144487671279, 0.9495099972971268, 0.20085517299359823, -0.0642414991767859], [-0.06167536426008202, -0.19155304542532242, 0.9795425362033394, 0.48596227502609984], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.3141522333276679, -0.06961288731962247, -0.9468169940459797, -0.2938321956293542], [0.9087902114310429, 0.2664212691161072, -0.3211231212007068, -0.4331291211280895], [0.2746064928262977, -0.9613395619039035, -0.020433326113291628, 0.21129924546928178], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.9294768118600414, -0.2133250040014556, -0.30094068997440704, 0.09088541126936657], [0.03416747134015019, 0.8621020367713744, -0.5055815088557608, -0.17617585560824017], [0.36729475917403626, 0.4596439065868564, 0.8085926285966625, 0.683460101273605], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[0.15075898698311613, -0.5221344230747579, -0.839432768054842, -0.28673946154775437], [0.9574559598971074, 0.28848238977311236, -0.007483024007904969, -0.10210505866141559], [0.2460687154055416, -0.8025917735880327, 0.5434120280850276, 0.3690234567125095], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.9560805951279769, 0.2370779477519957, 0.1723483168163672, 0.0605755152449563], [-0.10244528015735738, 0.8212175648774642, -0.5613436342475159, -0.031167658526970066], [-0.27461766183771463, 0.5190334843017649, 0.8094376949341786, 0.22780245105851482], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.8145533459253008, -0.19093795506603373, -0.547764131681784, -0.07208267548574171], [0.5170602584161668, 0.1890796370456493, -0.8348039171094567, -0.2944253030876043], [0.2629667960190113, -0.9632193873515458, -0.0552890226139773, 0.2989760361044501], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.5872689268857118, 0.023059489199478728, -0.8090633272324015, -0.21079955967604158], [-0.37780534668772414, 0.8918259239503066, -0.248816481335844, -0.19753344596857103], [0.7158060683793296, 0.45179063882294085, 0.5324536516386057, 0.5775651313270915], [0.0, 0.0, 0.0, 1.0]]}
diff --git a/pose_data1.txt b/pose_data1.txt
@@ -0,0 +1,10 @@
+{"base_to_hand": [[0.17774151597215418, 0.027123112596124526, -0.9837033548092731, -0.4857981053508395], [0.4469658541830354, 0.888337260947592, 0.10525414959306584, -0.15371028104492973], [0.8767151639468488, -0.45838984235607566, 0.1457713062657467, 0.23122432826380748], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.11826213827360499, 0.9796335631262081, 0.16227183565734493, 0.2928364919593094], [-0.9924795015328303, 0.1114128454569851, 0.05071111223709007, 0.05150344531582057], [0.031599140622800545, -0.1670486751334101, 0.9854421517512192, 0.44261983559519136], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[0.16305549784470044, 0.029694632109369797, -0.9861699313234542, -0.34851217751923846], [0.4468505243307062, 0.8889286915950967, 0.10064983936600382, -0.1279891571411036], [0.8796235066936249, -0.4570820605568749, 0.13167564842859736, 0.4216844665469352], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.12151363797707287, 0.9829236024280785, 0.1381869298283505, 0.15718250808385775], [-0.9919320609008131, 0.11518334817670817, 0.052948870241665905, -0.026212758435508333], [0.036127861030545896, -0.1435060559440326, 0.9889897823358671, 0.2632481099272212], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[0.18215078092652962, 0.12716502536776325, -0.9750128970075583, -0.24117518344723277], [0.42716516342634614, 0.8829057958855849, 0.19495455559330904, -0.08315321332884468], [0.8856359388387522, -0.45200266803953476, 0.10650151136178243, 0.3515706233673349], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.22209522033111995, 0.9641132168822371, 0.14546277234761826, 0.043606737981559035], [-0.974824844547098, 0.2165419942852208, 0.053160955265165497, 0.060263879040579574], [0.019754380774821056, -0.15360751851378673, 0.9879344587047463, 0.2953869388357075], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[0.4386033912879005, -0.07293887498608775, -0.8957159067843539, -0.10984289798519443], [0.4734601650961873, 0.865915333731176, 0.16132608864135267, -0.07679298031020725], [0.7638471949400902, -0.4948439706867109, 0.4143270537355311, 0.37343136836042823], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.01767725656635738, 0.8879355561501086, 0.45962806999212336, -0.10020524537506417], [-0.9995596554595747, 0.026651976664297805, -0.0130448195626452, 0.021383001423032795], [-0.023832955708921466, -0.4591950786586054, 0.888015692405204, 0.23525048695716236], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.15904532820210338, -0.05489440255239687, -0.9857439769765276, -0.3320835072837553], [0.37679769252301515, 0.9194995955066283, -0.1119999677342732, -0.20724297651254964], [0.9125393594176726, -0.38923912757013823, -0.12555803073488037, 0.30685325504662125], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.036891002218552016, 0.9826874862778756, -0.1815608941049277, 0.14237602366981206], [-0.9897232796323128, 0.0610464498262292, 0.1293103271880188, -0.04581850129267825], [0.13815528838654562, 0.17492465599932924, 0.9748407464885467, 0.4549021074584401], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.3035507678429248, -0.44014434065362273, -0.8450620632430252, -0.2123731038287327], [0.2741011440593278, 0.8090807821863978, -0.5198623382415913, -0.3282051137466162], [0.9125379412207134, -0.38943709028190254, -0.12495302535523875, 0.30703910934937023], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[-0.3555037516682463, 0.8864126935468919, -0.29646183442181284, 0.0061099609711423065], [-0.9270048033856568, -0.29382977601313565, 0.23307972290182322, -0.21268405017533104], [0.11949551058396371, 0.35768226045893703, 0.9261664124244967, 0.5534211748046899], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.3821042813550043, -0.6784920079168318, -0.6274112792763303, -0.059582166375444764], [0.14676896775061027, 0.6257557667700516, -0.7660865424085227, -0.29973758426607033], [0.9123898225405607, -0.38480945355722546, -0.13952238593642793, 0.2769598390462635], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[-0.6115120966777716, 0.7013469508807201, -0.36628596766334476, -0.18384523453628904], [-0.7904012735834379, -0.5202232489390454, 0.32347117024703076, -0.21262219685856257], [0.036315042811966014, 0.4873194288694238, 0.8724683328992197, 0.5944060685314653], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.02790126020726817, 0.2878073411196607, -0.9572818049438097, -0.469106952710388], [0.6677062664525528, 0.7180444306572529, 0.19641928963850433, -0.3358459217118114], [0.7439017821048953, -0.6337027142120613, -0.21220510969679263, 0.1882027101401396], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.3882345253606868, 0.9202737225274834, -0.048684997107637906, 0.2746417948913482], [-0.9082955674790176, 0.3731806963582827, -0.18903790615513255, -0.04427157886713884], [-0.15579831647335562, 0.11761140884691224, 0.980762173563557, 0.6178799101167423], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.2190947482407804, -0.02369757869066311, -0.9754157657417218, -0.35453380453829375], [0.6830920507787268, 0.7101067405763126, -0.1706858727340946, -0.5342319308430389], [0.6966941520181024, -0.703695134097349, -0.13939302992435298, 0.20666532437146024], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.07877397307051642, 0.975959581220361, -0.20321800360908648, 0.1718064315050996], [-0.9762599787699626, 0.03426441055592311, -0.213874739091196, -0.2866598473218598], [-0.20176995568904088, 0.2152413668267147, 0.9554894237969409, 0.7272356270075278], [0.0, 0.0, 0.0, 1.0]]}
+{"base_to_hand": [[-0.3292762749956133, 0.06735513143323231, -0.9418282332753808, -0.34542353074400217], [0.5066484325822309, 0.8543086064100042, -0.11603521351626521, -0.5186725320160682], [0.7967963983898158, -0.51538344102562, -0.31542892738239703, 0.05630563005475446], [0.0, 0.0, 0.0, 1.0]], "cam_to_tag": [[0.1646982676407498, 0.930574707553675, -0.3269635978171412, 0.15411519672002824], [-0.986220778989414, 0.16060615084232877, -0.03967668586367013, -0.17521637671037638], [0.015590244566686423, 0.32899297556787516, 0.9442036678076116, 0.8667226178488321], [0.0, 0.0, 0.0, 1.0]]}