Non-rigid registration of point clouds is still far from stable, especially for the largely deformed one. Sparse initial correspondences are often adopted to facilitate the process. However, there are few studies on how to build them automatically. Therefore, in this paper, we propose a robust method to compute such priors automatically, where a global and local combined strategy is adopted. These priors in different degrees of deformation are obtained by the locally geometrical-consistent point matches from the globally structural-consistent region correspondences. To further utilize the matches, this paper also proposes a novel registration method based on the Coherent Point Drift framework. This method takes both the spatial proximity and local structural consistency of the priors as supervision of the registration process and thus obtains a robust alignment for clouds with significantly different deformations. Qualitative and quantitative experiments demonstrate the advantages of the proposed method.

Non-rigid point cloud registration [

As an ill-posed problem, non-rigid point cloud registration often takes different prior assumptions to obtain a robust solution [

Therefore, a natural idea is to consider more global cues to filter out those inconsistent correspondences whose matching points are from different object parts. Directly separating semantically effective object parts is difficult because of the ambiguity in specifying their boundaries. However, the structures of the object are generally kept across different deformations, especially for the articulated objects (

Further, a more effective local-matching strategy is proposed so that globally consistent matches with strict local similarities can finally be taken as correct prior matches. Existing methods can be summarized as based on spatial (e.g., the Euclidean distance) or transformational (e.g., the descriptor similarity) proximity. However, these methods are good at slightly deformed regions and cannot cope with the largely deformed local ones. Therefore, a novel combined strategy with two local metrics (isometric consistency and structural consistency) are introduced to obtain the rich prior matches in different clouds (

The global and local combined strategy leads to sparse but strong prior correspondences for further registration. The robust Coherent Point Drift (CPD) [

In addition, in comparison with the recent developments in deep neural networks based ideas [

Consequently, our contributions are the following:

A novel global and local combined strategy to automatically estimate the robust prior matches for large deformed cloud pairs. Globally, a structure guided global matches are first obtained, so that correspondences not from the same regions are filtered out. Locally, a refinement strategy combining isometric consistency and structural consistency is proposed, so that effective prior correspondences with different degrees of deformation can be efficiently localized.

An improved CPD method with two additional supervised constraints from the estimated prior correspondences. The spatial proximity and local structural consistency constraints impose the spatial closeness and the local structure similarity of the priors, and their supervisions help make the registration successfully converge for large deformed clouds.

Generally, there are two types of registration, rigid and non-rigid. The former assumes a globally consistent rigid transformation between two clouds with the latter allowing local deformations between them. The non-rigid registration is more challenging than the rigid registration because it deals with a much larger solution space brought by the extra deformations. The related work on non-rigid point cloud registration is our main focus here. For more work on other work, such as the rigid ones, please refer to [

In the following, we will first show the general development of non-rigid registration and then focus on the sparse prior correspondences related work. Note that there are also some studies on building dense shape correspondences [

Traditional methods can be roughly classified into three types [

Each type can be further classified with different standards, such as optimization technique [

For the probabilistic based approaches, perhaps the most popular probabilistic method is the Coherent Point Drift method (CPD) [

One of the interesting extension to CPD is the Extended CPD (ECPD) method [

Local smooth regularizers have also been widely adopted [

Recently, deep learning based methods also attract attentions [

Existing methods for computing the prior correspondences can be roughly separate into three types: Manual specification, direct similarity and indirect similarity. Manual method relies on user specification [

Existing methods often resort directly to some spatial distance metrics. Some studies [

Some studies estimate the similarity of points with some description [

Recently, Li et al. [

Global structure we consider has also been taken for registering articulated objects [

Recently, Kleiman et al. [

The proposed registration method (

We first show how the initial putative matches are computed and then describe the global and local combined prior estimation idea.

The initial matches are created based on a feature signature of each point. The Heat Kernel Signature (HKS) [

When used as a feature descriptor, HKS restricts the heat kernel only to the temporal domain,

HKS is stable against noise and topology changes to some degree and also insensitive to the order of the eigenfunctions.

HKS can be adopted for multi-scale matching with the predefined time interval, which can be explained as that the heat distributes to the surface by that interval. Consequently, it can be extended as the feature descriptor of each point by sampling it into discrete function values according to different times. If that, the comparison between two points

In our method, the multi-scale signature of each point in both source and target clouds is computed. Experimentally, we set the set the time interval to be 15, i.e.,

As shown in

We compute the region correspondences by the structure-based method [

As shown in

Matches may be from the small or large deformed areas, which calls for different metrics to refine them from different deformed parts, so that a rich and evenly distributed matches can be obtained. For the small deformed areas, isometric consistency can be utilized to find matches, while for the large deformed areas, the structural consistency is more relaxed and can be adopted. Consequently, we take a combined local strategy where both the isometric and the structural consistencies are adopted.

Correspondences in the small-deformed areas generally share the same geodesic distances and thus isometric consistency can be used locally to filter out those significant from others. Here, the spectral pruning [

Assume two correspondences

In addition, both points of the related correspondences must also fall into the local geodesic discs around them. Assume the geodesic of

Consequently, the consistency for

Isometric is good for small deformed areas but may fail for large deformed areas. Therefore, a more relaxed filter is required so that matches with large deformations can be selected as priors. Here, we take the idea of LLE as the structural consistency measure, which is good at deformation measurement to compute the similarities between two points by their neighbors.

This consistency requires to first build a local area for computing the local structure relationship, which is fulfilled by first down sampling the source cloud and then taking each point as the center of each local area. Consequently, structural consistency for all matches starting from this central point is evaluated for finding the correct matches.

For a correspondence

Then

There is a problem with above

Another problem is the formation of the weight vector.

In this method, one of the two vectors fixes while the other varies. The Euclidean distance of the fixed vector to each permuted vector is computed and the smallest distance is finally obtained among all permutations.

After obtaining the matches according to both the isometric and structural consistencies, we can merge them to obtain the final prior matches. However, the structural consistency is more loose due to relaxed structure similarity for large deformation and thus may obtain more matches than the isometric consistency, as shown in

The prior matches obtained with above idea can be used to supervise the CPD registration. Assume two D-dimensional point sets

The energy model of our supervision based method consists of not only the traditionally whole cloud similarity term (data term) and motion coherent term from CPD but also the additional constraint brought by the prior correspondences.

We now discuss the three energy terms. The data term and motion coherent term are brought from ECPD, and thus are reviewed first briefly.

The overall process of the registration process in CPD can be formulated by the following energy function:

Here

The motion coherent term can be reformulated as

Consequently,

Two aspects are considered for the supervision of the prior correspondences: spatial proximity and neighboring structural consistency. The former can be formulated as the minimization of the point-to-point distances of the correspondences, while the latter can be represented by the similarities of transformations among the neighbors.

The nearest neighbors in

Putting the space proximity and structural consistency constraints together, we can obtain the following local prior constraint.

The optimization follows the basic idea of CPD where EM is adopted for solve the complex energy minimization problem. EM first guesses the values of parameters and uses the Bayes’ theorem to compute a posterior probability distribution

The object function in

Then,

Combining

The motion field can be obtained according to

For

In this section, both qualitative and quantitative experiments are presented based on the datasets: SCAPE [

Three methods are compared with ours in the experiment: CPD [

The quantitative experiments are applied to 20 cloud pairs with different deformations (

The overall sparsity of the prior matches can also be seen from

#Cloud pair | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 8 | 9 | 10 | 11 | 12 | 13 | 14 | 15 | 16 | 17 | 18 | 19 | 20 |
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|

Initial matches | 32960 | 32960 | 33720 | 33780 | 33620 | 33620 | 34080 | 33470 | 33220 | 33450 | 34850 | 33440 | 33470 | 33320 | 33880 | 36260 | 36650 | 36520 | 36280 | 33190 |

Final matches | 1411 | 1776 | 1728 | 1767 | 1312 | 1471 | 1597 | 1654 | 1293 | 1270 | 1369 | 1217 | 1328 | 1292 | 1500 | 1778 | 1673 | 1311 | 1232 | 1158 |

Ratio (%) | 4.2809 | 5.3883 | 5.1246 | 5.2309 | 3.9024 | 4.3754 | 4.6860 | 4.9417 | 3.8922 | 3.7967 | 3.9283 | 3.6394 | 3.9677 | 3.8776 | 4.4274 | 4.9035 | 4.5648 | 3.5898 | 3.3958 | 3.4890 |

The registration performance are experimented based on two metrics. One is matching accuracy

The other, average registration error

This paper proposed a novel method to automatically extract the sparse prior matches from global and local points. Globally, the region correspondences are created first to filter out matches whose points are from different object regions and obtain the global matches. Locally, isometric and structural consistencies are both considered to obtain rich prior matches in different deformations from the global ones. Then, a new prior-match based non-registration method is proposed where the prior correspondences are adopted into spatial proximity and neighboring structural consistency constraints to supervise the optimization of CPD. Experimental results demonstrate the advantages of the method.

Region matching is important for the extraction of local matches. However, our adopted region matching method is general and thus may not fit well with specifically deformed objects, such as stretch deformed ones. This matching method may also fall when there are highly symmetrical and drastically deformed clouds, such as human bodies rigged face to face. In the future, we will study more robust region matching methods or try other types of region corresponding methods [

This work is supported by

The authors declare that they have no conflicts of interest to report regarding the present study.