From 6b2d9b64f2136c90f2107e85b7fd4ba19894f5de Mon Sep 17 00:00:00 2001 From: Mark <155415633+mark0011astra@users.noreply.github.com> Date: Thu, 2 Jan 2025 12:47:13 +0900 Subject: [PATCH] Update README.md --- README.md | 140 ------------------------------------------------------ 1 file changed, 140 deletions(-) diff --git a/README.md b/README.md index 1f593ac..83c7e06 100644 --- a/README.md +++ b/README.md @@ -1,134 +1,3 @@ -# Hand Tracking Mouse Controller with Kalman Filter - -## Overview - -#### This project implements a system that controls the mouse cursor using real-time hand tracking with MediaPipe Hands. Kalman filter and win32api are used for high accuracy, low latency tracking and ease of operation. - -## Technical and Scientific Background - -### MediaPipe Hands - -[MediaPipe Hands](https://developers.google.com/mediapipe/solutions/vision/hand_landmarker) is a framework developed by Google that provides high-fidelity hand and finger tracking by employing machine learning (ML) to infer 21 3D landmarks of a hand from just a single frame. This system uses MediaPipe Hands to detect the hand position and its 21 landmarks (e.g., finger joints) in real-time from camera input. - -### Kalman Filter - -The Kalman filter is an algorithm that uses a series of measurements observed over time, containing statistical noise and other inaccuracies, and produces estimates of unknown variables that tend to be more accurate than those based on a single measurement alone, by estimating a joint probability distribution over the variables for each timeframe. The filter is named after Rudolf E. Kálmán, one of the primary developers of its theory. - -The Kalman filter has two distinct phases: **Predict** and **Update**. - -* **Predict phase:** The Kalman filter produces estimates of the current state variables, along with their uncertainties, based on the system model. -* **Update phase:** The next measurement is used to refine these predictions by a weighted average, with more weight being given to estimates with higher certainty. - -In this system, the hand position is considered the state, and the landmark positions obtained from MediaPipe Hands are the observed values. A constant velocity model is assumed, and the state is estimated using the following equations: - -**State Transition Matrix (F):** -content_copy -download -Use code with caution. -Markdown - -```math -Q = \begin{bmatrix} q1 & 0 & 0 & 0 \\ 0 & q2 & 0 & 0 \\ 0 & 0 & q3 & 0 \\ 0 & 0 & 0 & q4 \end{bmatrix} -``` -where `dt` is the time interval between frames. - -**Observation Matrix (H):** -content_copy -download -Use code with caution. - -H = | 1 0 0 0 | -| 0 1 0 0 | - -**Process Noise Covariance Matrix (Q):** -content_copy -download -Use code with caution. - -Q = | q1 0 0 0 | -| 0 q2 0 0 | -| 0 0 q3 0 | -| 0 0 0 q4 | - -where `q1`, `q2`, `q3`, and `q4` are parameters representing the magnitude of process noise. - -**Measurement Noise Covariance Matrix (R):** -content_copy -download -Use code with caution. - -R = | r1 0 | -| 0 r2 | - -where `r1` and `r2` are parameters representing the magnitude of measurement noise. - -By applying the Kalman filter, the noisy hand position data obtained from MediaPipe Hands is smoothed, resulting in smoother and more accurate mouse cursor movement. - -### Mouse Control - -The `win32api` library is used for mouse cursor control. The `win32api.SetCursorPos()` function moves the mouse cursor to the hand position estimated by the Kalman filter. - -### Sensitivity Adjustment - -The sensitivity of the mouse cursor (the amount of mouse cursor movement relative to hand movement) can be adjusted using the scaling factor `scaling_factor`. Increasing this factor increases the sensitivity. Fine-tuning of the sensitivity can also be achieved by adjusting the Kalman filter parameters `Q` and `R`. - -## Operating Environment - -* Windows 10/11 -* Python 3.7+ - -## Required Libraries - -* mediapipe -* opencv-python -* numpy -* filterpy -* pywin32 - -## Installation - -```bash -pip install mediapipe opencv-python numpy filterpy pywin32 -``` -Place your hand in front of the camera, and the mouse cursor will move according to your hand movements. - -### Configuration - -DEBUG (Default: False): Set to True to output debug information. - -scaling_factor (Default: 2.5): Adjusts the sensitivity of the mouse cursor. - -kf.Q (Default: np.eye(4) * 0.4): Process noise covariance matrix for the Kalman filter. - -kf.R (Default: np.eye(2) * 0.8): Measurement noise covariance matrix for the Kalman filter. - -These parameters can be configured at the beginning of main.py. - -### Troubleshooting - -Program does not start: Ensure that all required libraries are installed. - -Hand is not detected: Verify that the camera is connected correctly and that the operating environment meets the requirements of MediaPipe Hands. - -Mouse cursor does not move: Check if win32api is installed correctly. You may need to run the script with administrator privileges. - -Performance is slow: Lowering the values of min_detection_confidence and min_tracking_confidence may improve performance but could reduce accuracy. - -### Contribution - -Contributions to this project are welcome. Whether it's reporting bugs, suggesting features, or submitting pull requests, your contributions are appreciated. - -### License - -This project is released under the MIT License. See the LICENSE file for details. - -Acknowledgements - -This project utilizes the MediaPipe Hands and filterpy libraries. We thank the developers of these excellent libraries. - - - - # Hand Tracking Mouse Controller with Kalman Filter ## Overview @@ -249,15 +118,6 @@ This project utilizes the MediaPipe Hands and filterpy libraries. We thank the d - - - - - - - - - # Hand Tracking Mouse Controller with Kalman Filter ## 概要