With the help of surgical navigation system, doctors can operate on patients more intuitively and accurately. The positioning accuracy and real-time performance of surgical instruments are very important to the whole system. In this paper, we analyze and design the detection algorithm of surgical instrument location mark, and estimate the posture of surgical instrument. In addition, we optimized the pose by remapping. Finally, the algorithm of location mark detection proposed in this paper and the posture analysis data of surgical instruments are verified and analyzed through experiments. The final result shows a high accuracy.

With the development of the global economy, people's demand for health care is increasing, and countries have invested considerable funds in the medical field. At the same time, thanks to the development of computer technology and image processing technology, more and more technical means are used in the medical field [

Surgical navigation system technology has the advantages of high precision, high safety, low pain and minimally invasive, which makes the traditional surgical treatment schemes relying on doctors’ experience gradually eliminated. Its application field has gradually expanded and is widely used in orthopedic surgery [

Scholars have also done a lot of research on the localization algorithm of surgical navigation systems. Zhou et al. [

Besides, Wang et al. [

In addition, Chen et al. [

Furthermore, Zhang et al. [

The checkerboard is a frequently used pattern in camera calibration, an essential process to get intrinsic parameters for more accurate information from images. Yan et al. [

At present, the accuracy of surgical instrument posture positioning is required more and more in the field of surgical application. It is of great significance to study the factors affecting the accuracy of surgical instrument posture positioning, any lack of accuracy may have a huge impact on the surgical results. Therefore, the positioning accuracy in the current research still has room for further improvement.

This paper proposes a detection algorithm for surgical instrument position marking basedon the above analysis. We first design and analyze the markers, and then selects the markers that meet the accuracy and robustness for tracking and positioning. Then a detection algorithm suitable for the system is proposed and the experimental data is analyzed. Then the matching algorithm of surgical instruments is analyzed and designed, mainly consists of the matching algorithm of feature points and the matching algorithm of surgical instruments. Then, the gesture analysis algorithm of surgical instruments is analyzed and designed, mainly using the rigid body characteristics of surgical instruments to solve its gesture, and the algorithm of surgical instruments gesture optimization is studied. Finally, experimental data analysis, mainly black and white corner detection, surgical instrument positioning accuracy of experimental data analysis and comparison.

The data set selected in this paper is obtained by processing the time series data. Multiple data can be obtained for the same data series according to different time steps and mean square parameters. The data is classified according to the motion category of the time series. The sequence data is shown in the

Sport category | Data volume | |
---|---|---|

Training set | Plane random motion | 5231 |

Random motion | 10125 | |

Simulated surgical random motion | 20235 | |

Mixed motion | 20469 | |

Test set | Plane random motion | 1171 |

Random motion | 2296 | |

Simulated surgical random motion | 4312 | |

Mixed motion | 4482 |

In the specific experiment, the above data are sliced according to different demand steps, and the training data can be obtained by sampling. The plane random motion data in the training data is used to preliminarily train the model, and select the step size and sampling period, to find the preliminary parameter range suitable for the model. Some of the processed data in the above time series data are shown in

0.042 | 0.661 | 0.746 | −0.015 | 0.026 | 0.001 | 0.010 |

0.047 | 0.660 | 0.748 | −0.011 | −0.071 | 0.011 | 0.055 |

0.036 | 0.660 | 0.750 | −0.017 | −0.573 | 0.081 | 0.525 |

0.043 | 0.657 | 0.751 | −0.013 | −0.271 | 0.007 | 0.069 |

0.043 | 0.657 | 0.752 | −0.013 | −0.182 | 0.071 | −0.325 |

0.049 | 0.656 | 0.753 | −0.005 | −0.696 | 0.143 | −0.244 |

0.041 | 0.653 | 0.755 | −0.012 | −0.464 | −0.035 | 0.044 |

0.042 | 0.6526 | 0.756 | −0.006 | −0.131 | −0.079 | −0.067 |

0.043 | 0.648 | 0.759 | −0.008 | −0.242 | −0.188 | 0.154 |

0.034 | 0.648 | 0.760 | −0.004 | −0.147 | −0.227 | 0.227 |

In the process of surgical instrument matching, there are mainly two kinds of problems, that is, single surgical instrument matching and multi-surgical instrument matching. During the operation, doctors often need to use a variety of puncture tools, and there are situations where the surgeon has to cross surgical instruments between surgeries, which can lead to the problems of staggered positioning and disordered sequence of positioning markers. Therefore, this paper analyzes and studies the actual situation encountered in the process of manipulator matching, and puts forward a multi-surgical instrument matching algorithm suitable for this paper. The positioning mark points on multiple surgical instruments are imaged in one of the cameras, as shown in

Bouguet stereo correction is to make the baseline of the two cameras parallel to the camera’s imaging plane after correction, so that the poles are at infinity. At the same time, the optical axes of the left and right cameras are parallel, and the ordinates of the imaging points on the imaging plane of the left and right cameras are the same. After correction, feature point matching can be carried out only in the small range image area consistent with the ordinate of the point to be matched, which can greatly reduce the matching time, reduce the target matching error rate, and avoid the complex situation of multi-point corresponding pole line crossing in the opposite pole geometry. In this paper, Bouguet stereo correction method [

The principle of Bouguet stereo correction is to transform the whole transformation matrix of left and right cameras

Then, according to the spatial positioning principle [

In addition, this paper regularly sets the angle of the black-and-white grid on the surgical instrument. Among the four black-and-white grids, the angle directions of the three black-and-white grids except the origin are the same and perpendicular to the angle direction of the black edge grid at the origin. Therefore, some points in the set that may be matched into surgical instruments can be quickly screened, so as to avoid the direct violent matching between all points in the set and make the matching algorithm more efficient. The matching diagram of multiple surgical instruments obtained by the above method is shown in

The specific process of surgical instrument gesture resolution is to use the four positioning markers on the surgical instrument to obtain the known information of point set

Singular value decomposition (SVD) and quaternion method are often used to solve such problems [

Let the quaternion be

Quaternion conversion to rotation matrix:

Use the above method to analyze the posture of surgical instruments, as shown in the

SVD is used to solve the transformation matrix between the coordinates of the camera and the surgical instrument in the coordinate system, which is the preliminary solution calculated by using the rigid body information of the surgical instrument during calibration. In order to obtain the optimal solution of the posture of the surgical instrument, this paper remaps the positioning mark points on the surgical instrument back to the left and right camera imaging planes and takes the preliminary posture solution of the surgical instrument as the starting point, The sum of the squares of the distance between the location marks projected on the left and right cameras and the detected location marks is a loss function, which is continuously optimized and iterated. The schematic diagram of reprojection optimization is shown in

Then the optimization problem can be described as:

In this paper, for some pictures intercepted in the experiment of real-time recognition of black-and-white lattice corner detection in the video stream, the central blue point is the recognition positioning point, and the arrow represents the direction of black-and-white lattice corner defined in the way described in this paper. The image resolution in the experiment is 1280 × 960. A round piece with a black-and-white grid corner diameter of 25 mm is pasted on the surgical instrument. The identification picture is shown in

In feature point screening based on symmetry, the corner point score

The fracture diagram of cross area is shown in

The effect diagram of the improved algorithm after different

Compared with the common Fast, Harris and Shi-Tomasi corner detection algorithms, this paper tests and makes statistical analysis on 10 pictures in the calibrated pictures, and shows part of the screenshot area of one picture. The detection effect pictures are as follows in

In order to avoid the value of various algorithm thresholds, which can affect the experimental comparison, in the process of this experiment, when adjusting the threshold, the original algorithm is set to be equivalent to the threshold of this algorithm in proportion for comparison. And the other algorithms adjust the threshold from small to large intervals until the critical threshold of black-and-white grid corners can be fully identified, and fine tune around the critical threshold. It can be seen from the figure that the distribution near the corners in the Fast corner detection diagram is extremely uneven, and many non-intersecting corners will be identified. Compared with fast, Harris corner detection map is more evenly distributed near corners and can identify some non-intersecting corners. In the Shi-Tomasi corner detection diagram, the corner recognition is relatively uniform, and the number of recognitions at each position is relatively stable. Still, some non-intersecting corners can also be recognized.

The image resolution adopted in this experiment is

Algorithm | Average time/MS | Average number of corners |
---|---|---|

Fast | 7.641 | 145.289 |

Harris | 151.453 | 107.854 |

Shi-Tomasi | 143.482 | 86.149 |

CQRD | 26.699 | 429.391/70.553 (connected number) |

Ours |

After corner screening, it is compared with the existing positioning algorithms including black-and-white lattice to further verify the certainty of corner points of black-and-white lattice. The comparison algorithms include ASCD algorithm proposed by Da et al. [

Algorithm | Average time/MS | Average error/pixel |
---|---|---|

ASCD | 4489.136 | 0.109 |

AXDA | 2086.453 | 0.117 |

ACRSC | 1024.825 | 0.129 |

FTM | 219.754 | 0.134 |

CQRD | 31.699 | 0. 141 |

Ours |

We perform feature matching on the corner points of the surgical instrument, calculate the spatial coordinates, and then calculate the surgical instrument tip according to the rigid relationship of the surgical instrument. Print the square ABCD with the side length of 100 mm on the paper, stick it to the glass plate, and place the glass plate on the stable plane, with the needle tip at the four vertices of the square for the positioning test.

First group | Group 2 | Group 3 | Mean value | Variance | ||
---|---|---|---|---|---|---|

A | 194.780 | 194.857 | 195.031 | 194.889 | 0.105 | |

244.102 | 244.463 | 244.562 | 244.376 | 0.198 | ||

157.807 | 157.707 | 158.151 | 157.888 | 0.190 | ||

B | 108.903 | 108.933 | 109.115 | 108.982 | 0.091 | |

96.417 | 95.923 | 96.346 | 96.228 | 0.218 | ||

42.229 | 42.265 | 42.313 | 42.269 | 0.034 | ||

C | 21.116 | 21.221 | 21.408 | 21.248 | 0.121 | |

75.610 | 75.833 | 75.695 | 75.7127 | 0.092 | ||

830.652 | 830.458 | 830.498 | 830.536 | 0.084 | ||

D | 899.063 | 899.034 | 898.904 | 899.000 | 0.069 | |

944.299 | 944.101 | 944.291 | 944.230 | 0.092 | ||

876.532 | 876.447 | 876.457 | 876.479 | 0.038 |

The distance between adjacent points is shown in

First group | Group 2 | Group 3 | Mean value | Variance | |
---|---|---|---|---|---|

AB | 100.245 | 100.213 | 100.261 | 100.239 | 0.024 |

BC | 99.693 | 100.002 | 99.819 | 99.838 | 0.137 |

CD | 99.767 | 99.691 | 99.767 | 99.742 | 0.047 |

DA | 99.562 | 99.506 | 99.604 | 99.558 | 0.0465 |

It can be seen from the table that compared with the experimental side length of 100 mm, the maximum error does not exceed 0.5 mm. In order to avoid the problem of printing accuracy, the maximum error between the calculated data with the same side length does not exceed 0.755 mm, which fully meets the system requirements.

Let the tip of the surgical instrument move in a straight line about 20 cm away against the edge of the ruler on the plane of the glass plate, and fit the motion data of the tip of the surgical instrument with MATLAB to obtain the spatial straight-line equation as:

According to the analysis of the score and screening graph of the above original text algorithm, when

According to the scoring and screening graph of the original text algorithm, it is appropriate to select

To sum up, the improved algorithm not only requires relatively few candidate points for subsequent processing but also greatly improves the robustness of the improved algorithm, especially for light and tilt.

It can be seen from

It can be seen from

This paper studies the selection of surgical instruments, the recognition algorithm of black-and-white grid position marks and its influencing factors on the positioning accuracy of surgical instruments, and makes an experimental analysis. An improved black-and-white lattice detection algorithm based on symmetry is proposed. The symmetry operator is used for initial extraction, and then the corners are further extracted according to the regional and marginal diagonal points. Experiments show the stability and robustness of the improved algorithm. The matching method of feature points and the matching algorithm of surgical instruments are proposed. In addition, according to the rigid body characteristics of surgical instruments, the gesture parameters of surgical instruments are solved, and the estimated gesture accuracy is optimized by the remapping method. The experiment verifies the accuracy of the final results.

However, there are also imperfections in this article that can be improved, for example: In this paper, only the direction information of the corners of the black-and-white lattice is considered, and the size problem is not considered. At the same time, the surgical instruments designed in this paper can not rotate. Therefore, it is still necessary to further study the three-dimensional structure design of positioning marks of surgical instruments and the positioning algorithm with stable accuracy.

This work was supported by the

The authors declare that they have no conflicts of interest to report regarding the present study.