RINO
|
Development Story
The application was conceived with its goal being to monitor dynamic displacement of a target in real-time using smartphone technology. Many computer vision algorithms like corner-detection, line-detection, and template-matching were analyzed in the beginning but did not prove to be fast enough with the hardware capabilities of current smartphones. Many computer vision libraries and frameworks for iOS were explored like OpenCV. In the end, color-pattern matching used in tandem with Brad Larson's GPUImage were chosen because the algorithm's simplicity and reliance on the iPhone's GPU by using OpenGL shaders to perform color-pattern matching.
Despite color-pattern matching and GPU-based computer vision computation being key to the application's success, these two features could not alone achieve the real-time processing that was necessary. Introducing a crop filter that selectively processes areas in the incoming 1280x720p frame drastically improved processing speeds. The reduced number of pixels being processed allowed the application to process each frame, calculate the target's dynamic displacement, and update the user interface in real-time. The application's processing time could be reduced to below 7 miliseconds with the 720x100p crop filter (with iPhone 6), which is sufficient for 120 Hz sampling (see the right table).
Despite color-pattern matching and GPU-based computer vision computation being key to the application's success, these two features could not alone achieve the real-time processing that was necessary. Introducing a crop filter that selectively processes areas in the incoming 1280x720p frame drastically improved processing speeds. The reduced number of pixels being processed allowed the application to process each frame, calculate the target's dynamic displacement, and update the user interface in real-time. The application's processing time could be reduced to below 7 miliseconds with the 720x100p crop filter (with iPhone 6), which is sufficient for 120 Hz sampling (see the right table).
After the beta version of the app was completed, the user interface was in need of a makeover to improve the usability. The layout of the application was designed similar to the Camera app in the iPhone. A narrow controls menu with commonly used functionality like changing the sample rate and crop filter size and displaying the results of the color-pattern detection filter were added with additional buttons to return to the main menu and configure advanced settings. A preview of the 200x720p frame is shown with a thin green box (see the right picture) that indicates where the color-pattern detection filter will process. The full frame outside of the crop filter region is shown to provide the user with greater context as to where their device's rear-facing camera is pointing. Below the camera preview is a graph that shows the displacement of the target in millimeters with a record button that starts and stops displacement capturing to the right.
|
Functionalities
- Real-time Processing: Displacement calculations are made and the results are shown in real-time as the app tracks the target.
- Crop Filter: To further improve processing speeds, you can define the size of a crop filter which will determine which areas of the incoming 1280x720p frame will be analyzed for finding the target. (The green box in the screenshot above.)
- Target Detection Filter: RINO is configured to track the optimized blue-yellow/purple-green color-pattern by default. The filter's detection colors and search parameters can be configured to suit your collor-pattern's needs.
- Capture Rate: 30, 60, 120, & 240 Hz capture rates from the back-facing video camera are available. (Dependent upon iPhone model).
- Onboard Calibration: RINO autonomously converts and shows the captured movements of the target in engineering displacement unit (e.g. mm), no matter what the distance between the camera and target is.
- Data Exporting: After you have captured your data, you can email the data as a CSV file so further data processing and analyzing can be done.
Demonstration
Note: This demo video was made by recording iPhone6 screen directly while it measures the dynamic movement of a color-patteren target that was anmiated on a laptop's screen. See the top right of this web page for the color-pattern target animation.
Validation
Indoor and outdoor testing of the app were conducted. The color-pattern target was attached to a vertical plate on a motor-controlled shake-table. A fixed (high-precision) laser sensor was directed at vertical plate on the shake-table to validate thse results of the app. Mounted on the iPhone were 12x and 50x commercial telephoto lenses (see the right picture) for smartphones that allowed the target to be seen at long distances.
Below are pictures of the test setups and their results (both in indoor and outdoor). Click the pictures for more detail view. |
Application for wind tunnel testing
Reference
Min, J.H., Gelo, N.J., and Jo, H. (2015), “Non-contact and Real-time Dynamic Displacement Monitoring using Smartphone Technologies”, Journal of Life Cycle Reliability and Safety Engineering, 4 (2), 40-51.
Jaehong MIN, Nikolas Jorge Gelo, and Hongki Jo (2016), "US Patent: Measuring dynamic and absolute displacement", US 20160349040 A1
Jaehong MIN, Nikolas Jorge Gelo, and Hongki Jo (2016), "US Patent: Measuring dynamic and absolute displacement", US 20160349040 A1