Skip to content

API reference

kornia_moons.feature

Conversions between OpenCV and kornia local feature formats.

OpenCV cv2.KeyPoint lists ↔ kornia local affine frames (LAFs), cv2.DMatch lists ↔ kornia match tensors, and nn.Module wrappers that let OpenCV detectors and descriptors slot into kornia pipelines.

OpenCVDetectorKornia

Bases: Module

Wrap an OpenCV detector into a kornia-compatible detection module.

The wrapped detector runs on the numpy image under the hood; the module accepts and returns torch tensors, so it can slot into kornia pipelines.

Parameters:

Name Type Description Default
opencv_detector

Any OpenCV detector exposing detect (e.g. cv2.SIFT_create()).

 required
mrSize float

Measurement-region scale multiplier (6.0 for SIFT, 1.0 for ORB).

 6.0
make_upright

If True, deduplicate keypoints and zero their angles.

 False
max_kpts

Keep at most this many of the highest-response keypoints, applied whether or not make_upright is set. Non-positive values (the default -1) keep all keypoints.

 -1
Example

detector = OpenCVDetectorKornia(cv2.SIFT_create(500)) timg = kornia.image_to_tensor(img, False).float() / 255. lafs, resps = detector(timg)

Source code in kornia_moons/feature.py 
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
class OpenCVDetectorKornia(nn.Module):
    """Wrap an OpenCV detector into a kornia-compatible detection module.

    The wrapped detector runs on the numpy image under the hood; the module
    accepts and returns torch tensors, so it can slot into kornia pipelines.

    Args:
        opencv_detector: Any OpenCV detector exposing ``detect`` (e.g.
            ``cv2.SIFT_create()``).
        mrSize: Measurement-region scale multiplier (6.0 for SIFT, 1.0 for ORB).
        make_upright: If True, deduplicate keypoints and zero their angles.
        max_kpts: Keep at most this many of the highest-response keypoints,
            applied whether or not ``make_upright`` is set. Non-positive
            values (the default ``-1``) keep all keypoints.

    Example:
        >>> detector = OpenCVDetectorKornia(cv2.SIFT_create(500))
        >>> timg = kornia.image_to_tensor(img, False).float() / 255.
        >>> lafs, resps = detector(timg)
    """
    def __init__(self, opencv_detector, mrSize: float = 6.0, make_upright = False, max_kpts = -1):
        super().__init__()
        self.features = opencv_detector
        self.mrSize = mrSize
        self.make_upright = make_upright
        self.max_kpts = max_kpts

    def forward(self, x:torch.Tensor, mask=None):
        """Detect keypoints on a batched image tensor.

        Args:
            x: Image tensor of shape :math:`(1, C, H, W)`, values in
                :math:`[0, 1]` float or :math:`[0, 255]`.
            mask: Optional detection mask, :math:`(1, 1, H, W)` tensor or
                :math:`(H, W)` array; keypoints are detected where nonzero.

        Returns:
            LAFs of shape :math:`(1, N, 2, 3)` and responses :math:`(1, N, 1)`.
        """
        img_np = _image_to_numpy_uint8(x)
        kpts = self.features.detect(img_np, _mask_to_numpy(mask))
        if self.make_upright:
            kpts = make_keypoints_upright(kpts)
        if self.max_kpts > 0:
            kpts = sorted(kpts, key=lambda k: k.response, reverse=True)[:self.max_kpts]
        lafs, resp = laf_from_opencv_kpts(kpts, mrSize=self.mrSize, with_resp=True, device=x.device)
        return lafs, resp

forward(x, mask=None)

Detect keypoints on a batched image tensor.

Parameters:

Name Type Description Default
x Tensor

Image tensor of shape :math:(1, C, H, W), values in :math:[0, 1] float or :math:[0, 255].

 required
mask

Optional detection mask, :math:(1, 1, H, W) tensor or :math:(H, W) array; keypoints are detected where nonzero.

 None

Returns:

Type Description

LAFs of shape :math:(1, N, 2, 3) and responses :math:(1, N, 1).
Source code in kornia_moons/feature.py 
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
def forward(self, x:torch.Tensor, mask=None):
    """Detect keypoints on a batched image tensor.

    Args:
        x: Image tensor of shape :math:`(1, C, H, W)`, values in
            :math:`[0, 1]` float or :math:`[0, 255]`.
        mask: Optional detection mask, :math:`(1, 1, H, W)` tensor or
            :math:`(H, W)` array; keypoints are detected where nonzero.

    Returns:
        LAFs of shape :math:`(1, N, 2, 3)` and responses :math:`(1, N, 1)`.
    """
    img_np = _image_to_numpy_uint8(x)
    kpts = self.features.detect(img_np, _mask_to_numpy(mask))
    if self.make_upright:
        kpts = make_keypoints_upright(kpts)
    if self.max_kpts > 0:
        kpts = sorted(kpts, key=lambda k: k.response, reverse=True)[:self.max_kpts]
    lafs, resp = laf_from_opencv_kpts(kpts, mrSize=self.mrSize, with_resp=True, device=x.device)
    return lafs, resp

OpenCVFeatureKornia

Bases: Module

Wrap an OpenCV detect-and-describe pipeline for kornia.

The wrapped feature (e.g. cv2.SIFT_create()) runs detectAndCompute on the numpy image under the hood; the module accepts and returns torch tensors, so it can slot into kornia pipelines.

Parameters:

Name Type Description Default
opencv_detector

Any OpenCV feature exposing detectAndCompute.

 required
mrSize float

Measurement-region scale multiplier (6.0 for SIFT, 1.0 for ORB).

 6.0
Example

feature = OpenCVFeatureKornia(cv2.SIFT_create(500)) timg = kornia.image_to_tensor(img, False).float() / 255. lafs, resps, descs = feature(timg)

Source code in kornia_moons/feature.py 
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
class OpenCVFeatureKornia(nn.Module):
    """Wrap an OpenCV detect-and-describe pipeline for kornia.

    The wrapped feature (e.g. ``cv2.SIFT_create()``) runs ``detectAndCompute``
    on the numpy image under the hood; the module accepts and returns torch
    tensors, so it can slot into kornia pipelines.

    Args:
        opencv_detector: Any OpenCV feature exposing ``detectAndCompute``.
        mrSize: Measurement-region scale multiplier (6.0 for SIFT, 1.0 for ORB).

    Example:
        >>> feature = OpenCVFeatureKornia(cv2.SIFT_create(500))
        >>> timg = kornia.image_to_tensor(img, False).float() / 255.
        >>> lafs, resps, descs = feature(timg)
    """
    def __init__(self, opencv_detector, mrSize: float = 6.0):
        super().__init__()
        self.features = opencv_detector
        self.mrSize = mrSize

    def forward(self, x:torch.Tensor, mask=None):
        """Detect and describe keypoints on a batched image tensor.

        Args:
            x: Image tensor of shape :math:`(1, C, H, W)`, values in
                :math:`[0, 1]` float or :math:`[0, 255]`.
            mask: Optional detection mask, :math:`(1, 1, H, W)` tensor or
                :math:`(H, W)` array; keypoints are detected where nonzero.

        Returns:
            LAFs of shape :math:`(1, N, 2, 3)`, responses of shape
            :math:`(1, N, 1)`, and descriptors of shape :math:`(1, N, D)`.
        """
        img_np = _image_to_numpy_uint8(x)
        kpts, descs = self.features.detectAndCompute(img_np, _mask_to_numpy(mask))
        if descs is None:
            descs = np.zeros((0, self.features.descriptorSize()), dtype=np.float32)
        lafs, resp = laf_from_opencv_kpts(kpts, mrSize=self.mrSize, with_resp=True, device=x.device)
        return lafs, resp, torch.from_numpy(descs).to(device=x.device)[None]

forward(x, mask=None)

Detect and describe keypoints on a batched image tensor.

Parameters:

Name Type Description Default
x Tensor

Image tensor of shape :math:(1, C, H, W), values in :math:[0, 1] float or :math:[0, 255].

 required
mask

Optional detection mask, :math:(1, 1, H, W) tensor or :math:(H, W) array; keypoints are detected where nonzero.

 None

Returns:

Type Description

LAFs of shape :math:(1, N, 2, 3), responses of shape

math:(1, N, 1), and descriptors of shape :math:(1, N, D).
Source code in kornia_moons/feature.py 
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
def forward(self, x:torch.Tensor, mask=None):
    """Detect and describe keypoints on a batched image tensor.

    Args:
        x: Image tensor of shape :math:`(1, C, H, W)`, values in
            :math:`[0, 1]` float or :math:`[0, 255]`.
        mask: Optional detection mask, :math:`(1, 1, H, W)` tensor or
            :math:`(H, W)` array; keypoints are detected where nonzero.

    Returns:
        LAFs of shape :math:`(1, N, 2, 3)`, responses of shape
        :math:`(1, N, 1)`, and descriptors of shape :math:`(1, N, D)`.
    """
    img_np = _image_to_numpy_uint8(x)
    kpts, descs = self.features.detectAndCompute(img_np, _mask_to_numpy(mask))
    if descs is None:
        descs = np.zeros((0, self.features.descriptorSize()), dtype=np.float32)
    lafs, resp = laf_from_opencv_kpts(kpts, mrSize=self.mrSize, with_resp=True, device=x.device)
    return lafs, resp, torch.from_numpy(descs).to(device=x.device)[None]

OpenCVDetectorWithAffNetKornia

Bases: Module

Combine an OpenCV detector with kornia's AffNet affine-shape refinement.

The wrapped detector locates keypoints on the numpy image, and kornia's LAFAffNetShapeEstimator then refines the elliptical shape of each LAF. Note that AffNet's pretrained weights are downloaded on first use, and the original keypoint orientation is preserved after refinement.

Parameters:

Name Type Description Default
opencv_detector

Any OpenCV detector exposing detect.

 required
make_upright

If True, deduplicate keypoints and zero their angles.

 False
mrSize float

Measurement-region scale multiplier (6.0 for SIFT, 1.0 for ORB).

 6.0
max_kpts

Keep at most this many of the highest-response keypoints, applied whether or not make_upright is set. Non-positive values (the default -1) keep all keypoints.

 -1
Example

detector = OpenCVDetectorWithAffNetKornia(cv2.SIFT_create(500)) timg = kornia.image_to_tensor(img, False).float() / 255. lafs, resps = detector(timg)

Source code in kornia_moons/feature.py 
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
class OpenCVDetectorWithAffNetKornia(nn.Module):
    """Combine an OpenCV detector with kornia's AffNet affine-shape refinement.

    The wrapped detector locates keypoints on the numpy image, and kornia's
    ``LAFAffNetShapeEstimator`` then refines the elliptical shape of each
    LAF. Note that AffNet's pretrained weights are downloaded on first use,
    and the original keypoint orientation is preserved after refinement.

    Args:
        opencv_detector: Any OpenCV detector exposing ``detect``.
        make_upright: If True, deduplicate keypoints and zero their angles.
        mrSize: Measurement-region scale multiplier (6.0 for SIFT, 1.0 for ORB).
        max_kpts: Keep at most this many of the highest-response keypoints,
            applied whether or not ``make_upright`` is set. Non-positive
            values (the default ``-1``) keep all keypoints.

    Example:
        >>> detector = OpenCVDetectorWithAffNetKornia(cv2.SIFT_create(500))
        >>> timg = kornia.image_to_tensor(img, False).float() / 255.
        >>> lafs, resps = detector(timg)
    """
    def __init__(self, opencv_detector, make_upright = False, mrSize: float = 6.0, max_kpts = -1):
        super().__init__()
        self.features = opencv_detector
        self.mrSize = mrSize
        self.affnet = kornia.feature.LAFAffNetShapeEstimator(True).eval()
        self.make_upright = make_upright
        self.max_kpts = max_kpts

    def forward(self, x:torch.Tensor, mask=None):
        """Detect keypoints and refine their affine shape with AffNet.

        Args:
            x: Image tensor of shape :math:`(1, C, H, W)`, values in
                :math:`[0, 1]` float or :math:`[0, 255]`.
            mask: Optional detection mask, :math:`(1, 1, H, W)` tensor or
                :math:`(H, W)` array; keypoints are detected where nonzero.

        Returns:
            LAFs of shape :math:`(1, N, 2, 3)` and responses :math:`(1, N, 1)`.
        """
        self.affnet = self.affnet.to(x.device)
        img_np = _image_to_numpy_uint8(x)
        kpts = self.features.detect(img_np, _mask_to_numpy(mask))
        if self.make_upright:
            kpts = make_keypoints_upright(kpts)
        if self.max_kpts > 0:
            kpts = sorted(kpts, key=lambda k: k.response, reverse=True)[:self.max_kpts]
        lafs, resp = laf_from_opencv_kpts(kpts, mrSize=self.mrSize, with_resp=True, device=x.device)
        ori = kornia.feature.get_laf_orientation(lafs)
        lafs = self.affnet(lafs, x.mean(dim=1, keepdim=True))
        lafs = kornia.feature.set_laf_orientation(lafs, ori)
        return lafs, resp

forward(x, mask=None)

Detect keypoints and refine their affine shape with AffNet.

Parameters:

Name Type Description Default
x Tensor

Image tensor of shape :math:(1, C, H, W), values in :math:[0, 1] float or :math:[0, 255].

 required
mask

Optional detection mask, :math:(1, 1, H, W) tensor or :math:(H, W) array; keypoints are detected where nonzero.

 None

Returns:

Type Description

LAFs of shape :math:(1, N, 2, 3) and responses :math:(1, N, 1).
Source code in kornia_moons/feature.py 
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
def forward(self, x:torch.Tensor, mask=None):
    """Detect keypoints and refine their affine shape with AffNet.

    Args:
        x: Image tensor of shape :math:`(1, C, H, W)`, values in
            :math:`[0, 1]` float or :math:`[0, 255]`.
        mask: Optional detection mask, :math:`(1, 1, H, W)` tensor or
            :math:`(H, W)` array; keypoints are detected where nonzero.

    Returns:
        LAFs of shape :math:`(1, N, 2, 3)` and responses :math:`(1, N, 1)`.
    """
    self.affnet = self.affnet.to(x.device)
    img_np = _image_to_numpy_uint8(x)
    kpts = self.features.detect(img_np, _mask_to_numpy(mask))
    if self.make_upright:
        kpts = make_keypoints_upright(kpts)
    if self.max_kpts > 0:
        kpts = sorted(kpts, key=lambda k: k.response, reverse=True)[:self.max_kpts]
    lafs, resp = laf_from_opencv_kpts(kpts, mrSize=self.mrSize, with_resp=True, device=x.device)
    ori = kornia.feature.get_laf_orientation(lafs)
    lafs = self.affnet(lafs, x.mean(dim=1, keepdim=True))
    lafs = kornia.feature.set_laf_orientation(lafs, ori)
    return lafs, resp

to_numpy_image(img)

Load or convert an image into an RGB numpy array.

Parameters:

Name Type Description Default
img Union[str, array, Tensor]

Image file path, (B, C, H, W) / (C, H, W) tensor, or numpy array. Paths are read with OpenCV and converted to RGB.

 required

Returns:

Type Description

The image as an np.ndarray.

Raises:

Type Description
TypeError

If img is not a str, np.ndarray, or torch.Tensor.
FileNotFoundError

If img is a path that cannot be read.
Source code in kornia_moons/feature.py 
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
def to_numpy_image(img: Union[str, np.array, torch.Tensor]):
    """Load or convert an image into an RGB numpy array.

    Args:
        img: Image file path, `(B, C, H, W)` / `(C, H, W)` tensor, or
            numpy array. Paths are read with OpenCV and converted to RGB.

    Returns:
        The image as an ``np.ndarray``.

    Raises:
        TypeError: If ``img`` is not a str, np.ndarray, or torch.Tensor.
        FileNotFoundError: If ``img`` is a path that cannot be read.
    """
    if type(img) is str:
        img_bgr = cv2.imread(img)
        if img_bgr is None:
            raise FileNotFoundError(f'Could not read image: {img}')
        img_out = cv2.cvtColor(img_bgr, cv2.COLOR_BGR2RGB)
    elif isinstance(img, torch.Tensor):
        img_out = tensor_to_image(img)
    elif isinstance(img, np.ndarray):
        img_out = img
    else:
        raise TypeError('img should be str, np.array or torch.Tensor')
    return img_out

to_torch(x)

Convert a Python list or numpy array to a torch tensor, passing tensors through.

Parameters:

Name Type Description Default
x Union[List, array, Tensor]

List, numpy array, or torch tensor to convert.

 required

Returns:

Type Description

x as a torch.Tensor.

Raises:

Type Description
TypeError

If x is not a list, np.ndarray, or torch.Tensor.
Source code in kornia_moons/feature.py 
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
def to_torch(x: Union[List, np.array, torch.Tensor]):
    """Convert a Python list or numpy array to a torch tensor, passing tensors through.

    Args:
        x: List, numpy array, or torch tensor to convert.

    Returns:
        ``x`` as a ``torch.Tensor``.

    Raises:
        TypeError: If ``x`` is not a list, np.ndarray, or torch.Tensor.
    """
    if isinstance(x, list):
        x_out = torch.tensor(x)
    elif isinstance(x, torch.Tensor):
        x_out = x
    elif isinstance(x, np.ndarray):
        x_out = torch.from_numpy(x)
    else:
        raise TypeError('img should be List, np.array or torch.Tensor')
    return x_out

to_np(array)

Convert a tensor or sequence to a numpy array.

Parameters:

Name Type Description Default
array Union[List, Tuple, ndarray, Tensor]

List, tuple, numpy array, or torch tensor to convert. Tensors are detached and moved to CPU before conversion.

 required

Returns:

Type Description

array as an np.ndarray.
Source code in kornia_moons/feature.py 
78
79
80
81
82
83
84
85
86
87
88
89
90
def to_np(array: Union[List, Tuple, np.ndarray, torch.Tensor]):
    """Convert a tensor or sequence to a numpy array.

    Args:
        array: List, tuple, numpy array, or torch tensor to convert.
            Tensors are detached and moved to CPU before conversion.

    Returns:
        ``array`` as an ``np.ndarray``.
    """
    if isinstance(array, torch.Tensor):
        return array.detach().cpu().numpy()
    return np.asarray(array)

laf_from_opencv_kpts(kpts, mrSize=6.0, device=torch.device('cpu'), with_resp=False)

Convert OpenCV keypoints into kornia local affine frames (LAFs).

Parameters:

Name Type Description Default
kpts List[KeyPoint]

List of N OpenCV keypoints. Keypoints with angle < 0 (OpenCV's "orientation not computed" sentinel, e.g. from GFTT or FAST) become upright LAFs.

 required
mrSize float

Measurement-region scale multiplier applied to the keypoint size. Use 6.0 for SIFT-like detectors and 1.0 for ORB-like detectors, matching the OpenCV description-region conventions.

 6.0
device device

Device to place the output tensors on.

 device('cpu')
with_resp bool

If True, also return the keypoint responses.

 False

Returns:

Type Description
Union[Tensor, Tuple[Tensor, Tensor]]

LAFs of shape :math:(1, N, 2, 3), and, if with_resp is True,
Union[Tensor, Tuple[Tensor, Tensor]]

a tuple of the LAFs and responses of shape :math:(1, N, 1).
Example

img = cv2.imread('data/strahov.png') kps, descs = cv2.ORB_create(500).detectAndCompute(img, None) lafs, resp = laf_from_opencv_kpts(kps, mrSize=1.0, with_resp=True)

Source code in kornia_moons/feature.py 
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
def laf_from_opencv_kpts(kpts: List[cv2.KeyPoint], 
                         mrSize: float=6.0,
                         device: torch.device=torch.device('cpu'),
                         with_resp: bool = False) -> Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]:
    """Convert OpenCV keypoints into kornia local affine frames (LAFs).

    Args:
        kpts: List of N OpenCV keypoints. Keypoints with ``angle < 0``
            (OpenCV's "orientation not computed" sentinel, e.g. from GFTT
            or FAST) become upright LAFs.
        mrSize: Measurement-region scale multiplier applied to the keypoint
            size. Use 6.0 for SIFT-like detectors and 1.0 for ORB-like
            detectors, matching the OpenCV description-region conventions.
        device: Device to place the output tensors on.
        with_resp: If True, also return the keypoint responses.

    Returns:
        LAFs of shape :math:`(1, N, 2, 3)`, and, if ``with_resp`` is True,
        a tuple of the LAFs and responses of shape :math:`(1, N, 1)`.

    Example:
        >>> img = cv2.imread('data/strahov.png')
        >>> kps, descs = cv2.ORB_create(500).detectAndCompute(img, None)
        >>> lafs, resp = laf_from_opencv_kpts(kps, mrSize=1.0, with_resp=True)
    """
    N = len(kpts)
    xy = torch.tensor([(x.pt[0], x.pt[1]) for x in kpts ], device=device, dtype=torch.float).view(1, N, 2)
    scales = torch.tensor([(mrSize * x.size) for x in kpts ], device=device, dtype=torch.float).view(1, N, 1, 1)
    # angle < 0 (OpenCV's -1) means "orientation not computed" -> upright
    angles = torch.tensor([(-x.angle if x.angle >= 0 else 0.0) for x in kpts ], device=device, dtype=torch.float).view(1, N, 1)
    laf = kornia.feature.laf_from_center_scale_ori(xy, scales, angles).reshape(1, -1, 2, 3)
    if not with_resp:
        return laf.reshape(1, -1, 2, 3)
    resp = torch.tensor([x.response for x in kpts], device=device, dtype=torch.float).view(1, N, 1)
    return laf, resp

opencv_kpts_from_laf(lafs, mrSize=1.0, resps=None)

Convert kornia local affine frames (LAFs) back to OpenCV keypoints.

Parameters:

Name Type Description Default
lafs Tensor

LAFs of shape :math:(1, N, 2, 3).

 required
mrSize float

Measurement-region scale multiplier to divide out of the LAF scale. Must match the mrSize used to create the LAFs (6.0 for SIFT-like, 1.0 for ORB-like detectors).

 1.0
resps Optional[Tensor]

Optional keypoint responses of shape :math:(1, N, 1).

 None

Returns:

Type Description
List[KeyPoint]

List of N OpenCV keypoints.
Note

A LAF does not store the OpenCV octave and class_id fields, so keypoints converted back get the defaults (octave=0, class_id=-1). OpenCV descriptor extractors still work on such keypoints.

Example

lafs, resp = laf_from_opencv_kpts(kps, mrSize=1.0, with_resp=True) kps_back = opencv_kpts_from_laf(lafs, 1.0, resp)

Source code in kornia_moons/feature.py 
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
def opencv_kpts_from_laf(lafs: torch.Tensor,
                         mrSize: float = 1.0, 
                         resps: Optional[torch.Tensor] = None) -> List[cv2.KeyPoint]:
    """Convert kornia local affine frames (LAFs) back to OpenCV keypoints.

    Args:
        lafs: LAFs of shape :math:`(1, N, 2, 3)`.
        mrSize: Measurement-region scale multiplier to divide out of the LAF
            scale. Must match the ``mrSize`` used to create the LAFs (6.0
            for SIFT-like, 1.0 for ORB-like detectors).
        resps: Optional keypoint responses of shape :math:`(1, N, 1)`.

    Returns:
        List of N OpenCV keypoints.

    Note:
        A LAF does not store the OpenCV ``octave`` and ``class_id`` fields,
        so keypoints converted back get the defaults (``octave=0``,
        ``class_id=-1``). OpenCV descriptor extractors still work on such
        keypoints.

    Example:
        >>> lafs, resp = laf_from_opencv_kpts(kps, mrSize=1.0, with_resp=True)
        >>> kps_back = opencv_kpts_from_laf(lafs, 1.0, resp)
    """
    XY = kornia.feature.get_laf_center(lafs)
    S = kornia.feature.get_laf_scale(lafs)
    Ang = kornia.feature.get_laf_orientation(lafs)
    if resps is not None:
        assert resps.shape[:2] == lafs.shape[:2]
        cv_kpts = [cv2.KeyPoint(xy[0].item(), xy[1].item(), s.item()/mrSize, -a.item(), r.item()) 
                   for xy, s, a, r in zip(XY.view(-1,2), S.view(-1, 1), Ang.view(-1, 1), resps.view(-1, 1))]
    else:
        cv_kpts = [cv2.KeyPoint(xy[0].item(), xy[1].item(), s.item()/ mrSize, -a.item()) 
                   for xy, s, a in zip(XY.view(-1,2), S.view(-1, 1), Ang.view(-1, 1))]        
    return cv_kpts

laf_from_opencv_ORB_kpts(kpts, device=torch.device('cpu'), with_resp=False)

Convert ORB keypoints into kornia LAFs using the ORB mrSize convention.

Parameters:

Name Type Description Default
kpts List[KeyPoint]

List of N OpenCV ORB keypoints.

 required
device device

Device to place the output tensors on.

 device('cpu')
with_resp bool

If True, also return the keypoint responses.

 False

Returns:

Type Description
Union[Tensor, Tuple[Tensor, Tensor]]

LAFs of shape :math:(1, N, 2, 3), and, if with_resp is True,
Union[Tensor, Tuple[Tensor, Tensor]]

a tuple of the LAFs and responses of shape :math:(1, N, 1).
Source code in kornia_moons/feature.py 
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
def laf_from_opencv_ORB_kpts(kpts: List[cv2.KeyPoint], 
                             device: torch.device=torch.device('cpu'),
                             with_resp: bool = False) -> Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]:
    """Convert ORB keypoints into kornia LAFs using the ORB `mrSize` convention.

    Args:
        kpts: List of N OpenCV ORB keypoints.
        device: Device to place the output tensors on.
        with_resp: If True, also return the keypoint responses.

    Returns:
        LAFs of shape :math:`(1, N, 2, 3)`, and, if ``with_resp`` is True,
        a tuple of the LAFs and responses of shape :math:`(1, N, 1)`.
    """
    return laf_from_opencv_kpts(kpts, 1.0, device, with_resp)

laf_from_opencv_SIFT_kpts(kpts, device=torch.device('cpu'), with_resp=False)

Convert SIFT keypoints into kornia LAFs using the SIFT mrSize convention.

Parameters:

Name Type Description Default
kpts List[KeyPoint]

List of N OpenCV SIFT keypoints.

 required
device device

Device to place the output tensors on.

 device('cpu')
with_resp bool

If True, also return the keypoint responses.

 False

Returns:

Type Description
Union[Tensor, Tuple[Tensor, Tensor]]

LAFs of shape :math:(1, N, 2, 3), and, if with_resp is True,
Union[Tensor, Tuple[Tensor, Tensor]]

a tuple of the LAFs and responses of shape :math:(1, N, 1).
Source code in kornia_moons/feature.py 
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
def laf_from_opencv_SIFT_kpts(kpts: List[cv2.KeyPoint], 
                              device: torch.device=torch.device('cpu'),
                              with_resp: bool = False) -> Union[torch.Tensor, Tuple[torch.Tensor, torch.Tensor]]:
    """Convert SIFT keypoints into kornia LAFs using the SIFT `mrSize` convention.

    Args:
        kpts: List of N OpenCV SIFT keypoints.
        device: Device to place the output tensors on.
        with_resp: If True, also return the keypoint responses.

    Returns:
        LAFs of shape :math:`(1, N, 2, 3)`, and, if ``with_resp`` is True,
        a tuple of the LAFs and responses of shape :math:`(1, N, 1)`.
    """
    return laf_from_opencv_kpts(kpts, 6.0, device, with_resp)

opencv_SIFT_kpts_from_laf(lafs, resps=None)

Convert kornia LAFs to OpenCV keypoints using the SIFT mrSize convention.

Parameters:

Name Type Description Default
lafs

LAFs of shape :math:(1, N, 2, 3).

 required
resps Optional[Tensor]

Optional keypoint responses of shape :math:(1, N, 1).

 None

Returns:

Type Description

List of N OpenCV keypoints.
Source code in kornia_moons/feature.py 
199
200
201
202
203
204
205
206
207
208
209
def opencv_SIFT_kpts_from_laf(lafs, resps: Optional[torch.Tensor] = None):
    """Convert kornia LAFs to OpenCV keypoints using the SIFT `mrSize` convention.

    Args:
        lafs: LAFs of shape :math:`(1, N, 2, 3)`.
        resps: Optional keypoint responses of shape :math:`(1, N, 1)`.

    Returns:
        List of N OpenCV keypoints.
    """
    return opencv_kpts_from_laf(lafs, 6.0, resps);

opencv_ORB_kpts_from_laf(lafs, resps=None)

Convert kornia LAFs to OpenCV keypoints using the ORB mrSize convention.

Parameters:

Name Type Description Default
lafs

LAFs of shape :math:(1, N, 2, 3).

 required
resps Optional[Tensor]

Optional keypoint responses of shape :math:(1, N, 1).

 None

Returns:

Type Description

List of N OpenCV keypoints.
Source code in kornia_moons/feature.py 
211
212
213
214
215
216
217
218
219
220
221
def opencv_ORB_kpts_from_laf(lafs, resps: Optional[torch.Tensor] = None):
    """Convert kornia LAFs to OpenCV keypoints using the ORB `mrSize` convention.

    Args:
        lafs: LAFs of shape :math:`(1, N, 2, 3)`.
        resps: Optional keypoint responses of shape :math:`(1, N, 1)`.

    Returns:
        List of N OpenCV keypoints.
    """
    return opencv_kpts_from_laf(lafs, 1.0, resps);

cv2_matches_from_kornia(match_dists, match_idxs)

Convert kornia match distances and indexes to a list of cv2.DMatch.

Parameters:

Name Type Description Default
match_dists Tensor

Match distances of shape :math:(N, 1).

 required
match_idxs Tensor

Match indexes (query, train) of shape :math:(N, 2).

 required

Returns:

Type Description
List[DMatch]

List of N cv2.DMatch objects.
Example

dists, idxs = kornia.feature.match_nn(descs1, descs2) cv2_matches = cv2_matches_from_kornia(dists, idxs)

Source code in kornia_moons/feature.py 
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
def cv2_matches_from_kornia(match_dists: torch.Tensor, match_idxs: torch.Tensor) -> List[cv2.DMatch]:
    """Convert kornia match distances and indexes to a list of cv2.DMatch.

    Args:
        match_dists: Match distances of shape :math:`(N, 1)`.
        match_idxs: Match indexes (query, train) of shape :math:`(N, 2)`.

    Returns:
        List of N ``cv2.DMatch`` objects.

    Example:
        >>> dists, idxs = kornia.feature.match_nn(descs1, descs2)
        >>> cv2_matches = cv2_matches_from_kornia(dists, idxs)
    """
    return [cv2.DMatch(idx[0].item(), idx[1].item(), d.item()) for idx, d in zip (match_idxs, match_dists)]

kornia_matches_from_cv2(cv2_matches, device=torch.device('cpu'))

Convert a list of cv2.DMatch to kornia match distance and index tensors.

Parameters:

Name Type Description Default
cv2_matches

List of N cv2.DMatch objects, or the nested list returned by cv2.BFMatcher.knnMatch / radiusMatch, which is flattened (N is then the total number of matches).

 required
device

Device to place the output tensors on.

 device('cpu')

Returns:

Type Description

Tuple of match distances of shape :math:(N, 1) (float) and match

indexes (query, train) of shape :math:(N, 2) (long).
Example

cv2_matches = cv2_matches_from_kornia(match_dists, match_idxs) match_dists_back, match_idxs_back = kornia_matches_from_cv2(cv2_matches)

Source code in kornia_moons/feature.py 
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
def kornia_matches_from_cv2(cv2_matches, device=torch.device('cpu')):
    """Convert a list of cv2.DMatch to kornia match distance and index tensors.

    Args:
        cv2_matches: List of N ``cv2.DMatch`` objects, or the nested list
            returned by ``cv2.BFMatcher.knnMatch`` / ``radiusMatch``, which
            is flattened (N is then the total number of matches).
        device: Device to place the output tensors on.

    Returns:
        Tuple of match distances of shape :math:`(N, 1)` (float) and match
        indexes (query, train) of shape :math:`(N, 2)` (long).

    Example:
        >>> cv2_matches = cv2_matches_from_kornia(match_dists, match_idxs)
        >>> match_dists_back, match_idxs_back = kornia_matches_from_cv2(cv2_matches)
    """
    if len(cv2_matches) > 0 and isinstance(cv2_matches[0], (tuple, list)):
        cv2_matches = [m for group in cv2_matches for m in group]
    num_matches = len(cv2_matches)
    match_dists = torch.zeros(num_matches, 1, device=device, dtype=torch.float)
    match_idxs = torch.zeros(num_matches, 2, device=device, dtype=torch.long)
    for i, m in enumerate(cv2_matches):
        match_dists[i, 0] = m.distance
        match_idxs[i, 0] = m.queryIdx
        match_idxs[i, 1] = m.trainIdx
    return match_dists, match_idxs

make_keypoints_upright(kpts)

Deduplicate keypoints by response and zero their orientations.

Keypoints are considered duplicates when they share the same response as the previous keypoint in the input list. Note that this mutates the input keypoints in place (setting angle = 0).

Parameters:

Name Type Description Default
kpts

List of OpenCV keypoints, assumed sorted by response.

 required

Returns:

Type Description

The deduplicated, upright keypoints sorted by response, descending.
Source code in kornia_moons/feature.py 
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
def make_keypoints_upright(kpts):
    """Deduplicate keypoints by response and zero their orientations.

    Keypoints are considered duplicates when they share the same response
    as the previous keypoint in the input list. Note that this mutates the
    input keypoints in place (setting ``angle = 0``).

    Args:
        kpts: List of OpenCV keypoints, assumed sorted by response.

    Returns:
        The deduplicated, upright keypoints sorted by response, descending.
    """
    unique_kp = []
    for i, kk in enumerate(kpts):
        if i > 0:
            if kk.response == kpts[i - 1].response:
                continue
        kk.angle = 0
        unique_kp.append(kk)
    top_resps = np.array([kk.response for kk in unique_kp])
    idxs = np.argsort(top_resps)[::-1]
    return [unique_kp[i] for i in idxs]

kornia_moons.viz

Visualization helpers for local features and matches.

Drawing kornia LAFs over images, tentative/inlier matches between image pairs (with optional epipolar lines or reprojected corners), plain point matches from detector-free matchers such as LoFTR, epipolar errors, and SOLD2-style line segments.

visualize_LAF(img, LAF, img_idx=0, color='r', linewidth=1, draw_ori=True, fig=None, ax=None, return_fig_ax=False, **kwargs)

Draw local affine frames (LAFs) over an image.

Parameters:

Name Type Description Default
img

Batched image tensor of shape :math:(B, C, H, W).

 required
LAF

LAFs of shape :math:(B, N, 2, 3).

 required
img_idx

Index of the image in the batch to draw on.

 0
color

Matplotlib color for the LAF outlines.

 'r'
linewidth

Line width of the LAF outlines.

 1
draw_ori

If True, also draw the orientation line of each LAF.

 True
fig

Optional existing matplotlib figure to draw on.

 None
ax

Optional existing matplotlib axes to draw on.

 None
return_fig_ax

If True, return the figure and axes instead of None.

 False
**kwargs

Extra keyword arguments forwarded to plt.subplots.

 {}

Returns:

Type Description

Tuple of (fig, ax) if return_fig_ax is True, otherwise None.
Example

lafs, resp = laf_from_opencv_kpts(kps, mrSize=1.0, with_resp=True) visualize_LAF(image_to_tensor(img, False), lafs, 0, 'y', draw_ori=False)

Source code in kornia_moons/viz.py 
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
def visualize_LAF(img, LAF, img_idx = 0, color='r', linewidth=1,
                  draw_ori = True, fig=None,
                  ax = None, return_fig_ax = False, **kwargs):
    """Draw local affine frames (LAFs) over an image.

    Args:
        img: Batched image tensor of shape :math:`(B, C, H, W)`.
        LAF: LAFs of shape :math:`(B, N, 2, 3)`.
        img_idx: Index of the image in the batch to draw on.
        color: Matplotlib color for the LAF outlines.
        linewidth: Line width of the LAF outlines.
        draw_ori: If True, also draw the orientation line of each LAF.
        fig: Optional existing matplotlib figure to draw on.
        ax: Optional existing matplotlib axes to draw on.
        return_fig_ax: If True, return the figure and axes instead of None.
        **kwargs: Extra keyword arguments forwarded to ``plt.subplots``.

    Returns:
        Tuple of ``(fig, ax)`` if ``return_fig_ax`` is True, otherwise None.

    Example:
        >>> lafs, resp = laf_from_opencv_kpts(kps, mrSize=1.0, with_resp=True)
        >>> visualize_LAF(image_to_tensor(img, False), lafs, 0, 'y', draw_ori=False)
    """
    from kornia_moons.feature import to_numpy_image
    x, y = kornia.feature.laf.get_laf_pts_to_draw(kornia.feature.laf.scale_laf(LAF, 0.5), img_idx)
    if not draw_ori:
        x= x[1:]
        y= y[1:]
    if (fig is None and ax is None):
        fig, ax = plt.subplots(1,1, **kwargs)
    if (fig is not None and ax is None):
        ax = fig.add_axes([0, 0, 1, 1])

    ax.imshow(to_numpy_image(img[img_idx]))
    ax.plot(x, y, color, linewidth=linewidth)
    if return_fig_ax : return fig, ax
    return

epilines_to_start_end_points(epi, h, w)

Clip epipolar lines to their image-boundary start/end points.

Parameters:

Name Type Description Default
epi

Epipolar line coefficients (a, b, c) of shape :math:(N, 3).

 required
h

Image height.

 required
w

Image width.

 required

Returns:

Type Description

Stacked start and end points of shape :math:(2, N, 2) for an

h x w image, clipped to the image boundary.
Source code in kornia_moons/viz.py 
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
def epilines_to_start_end_points(epi, h, w):
    """Clip epipolar lines to their image-boundary start/end points.

    Args:
        epi: Epipolar line coefficients ``(a, b, c)`` of shape :math:`(N, 3)`.
        h: Image height.
        w: Image width.

    Returns:
        Stacked start and end points of shape :math:`(2, N, 2)` for an
        ``h`` x ``w`` image, clipped to the image boundary.
    """
    num = len(epi)
    zeros = torch.zeros(num, device=epi.device, dtype=epi.dtype)
    ones = torch.ones(num, device=epi.device, dtype=epi.dtype)
    eps = 1e-8


    b = -epi[:,2] / (epi[:,1] + eps)
    k = -epi[:,0] / (epi[:,1] + eps)
    k_positive = k >= 0
    k_negative = k < 0


    # We find the points of crossing x = 0, x = w, y = 0, y = h
    p1 = torch.stack([zeros,  b], axis=-1)
    p2 = torch.stack([w*ones, k*w + b], axis =-1)
    p3 = torch.stack([-b/(k + eps), zeros], axis=-1)
    p4 = torch.stack([(h - b)/(k + eps), ones*h], axis=-1)

    # Now we select those, which are inside of image
    p1_is_start = (p1[:, 1:2] <= h) & (p1[:, 1:2] >= 0)
    p3_is_start = (k_positive * (~p1_is_start.view(-1))).view(-1,1)
    p4_is_start = (k_negative * (~p1_is_start.view(-1))).view(-1,1)
    none_is_start = ~(p1_is_start | p3_is_start |  p4_is_start)


    p2_is_fin  = (p2[:, 1:2] <= h) & (p2[:, 1:2] >= 0)
    p4_is_fin  = (k_positive * (~p2_is_fin.view(-1))).view(-1,1)
    p3_is_fin  = (k_negative * (~p2_is_fin.view(-1))).view(-1,1)


    start_lines = p1 * p1_is_start.float() + p3 * p3_is_start.float() + p4 * p4_is_start.float()
    fin_lines   = p2 * p2_is_fin.float()   + p4 * p4_is_fin.float() + p3 * p3_is_fin.float()
    return torch.stack ([start_lines, fin_lines])

draw_LAF_matches(lafs1, lafs2, tent_idxs, img1, img2, inlier_mask=None, draw_dict={'inlier_color': (0.2, 1, 0.2), 'tentative_color': (0.8, 0.8, 0), 'feature_color': (0.2, 0.5, 1), 'vertical': False}, Fm=None, H=None, fig=None, ax=None, return_fig_ax=False)

This function draws LAFs, tentative matches, inliers epipolar lines (if F is provided), and image1 corners reprojection into image 2 (if H is provided)

Parameters:

Name Type Description Default
lafs1

LAFs of image 1, shape :math:(1, N1, 2, 3).

 required
lafs2

LAFs of image 2, shape :math:(1, N2, 2, 3).

 required
tent_idxs

Tentative match indexes (query, train) of shape :math:(M, 2).

 required
img1

First image, path/tensor/numpy array.

 required
img2

Second image, path/tensor/numpy array.

 required
inlier_mask

Optional boolean array/list of length M marking inliers among tent_idxs.

 None
draw_dict

Drawing options. Keys: inlier_color (RGB tuple or None to skip drawing inliers), tentative_color (RGB tuple or None to skip drawing tentative matches), feature_color (RGB tuple or None to skip drawing all detected LAFs), vertical (bool, stack images vertically instead of horizontally).

 {'inlier_color': (0.2, 1, 0.2), 'tentative_color': (0.8, 0.8, 0), 'feature_color': (0.2, 0.5, 1), 'vertical': False}
Fm Optional[array]

Optional fundamental matrix of shape :math:(3, 3) used to draw inlier epipolar lines.

 None
H Optional[array]

Optional homography of shape :math:(3, 3) used to draw the reprojection of image 1's corners into image 2.

 None
fig

Optional existing matplotlib figure to draw on.

 None
ax Optional

Optional existing matplotlib axes to draw on.

 None
return_fig_ax

If True, return the figure and axes instead of None.

 False

Returns:

Type Description

Tuple of (fig, ax) if return_fig_ax is True, otherwise None.
Example

match_dists, match_idxs = kornia.feature.match_snn(descs1, descs2, 0.98) draw_LAF_matches(lafs1, lafs2, match_idxs, img1, img2, mask, ... draw_dict={"inlier_color": (0.2, 1, 0.2), ... "tentative_color": (0.8, 0.8, 0), ... "feature_color": None, "vertical": False}, H=H)

Source code in kornia_moons/viz.py 
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
def draw_LAF_matches(lafs1, lafs2, tent_idxs,  
                     img1, img2, inlier_mask = None, 
                        draw_dict={"inlier_color": (0.2, 1, 0.2),
                               "tentative_color": (0.8, 0.8, 0), 
                               "feature_color": (0.2, 0.5, 1),
                                  "vertical": False}, 
                        Fm: Optional[np.array] = None, H: Optional[np.array] = None,
                        fig = None, ax: Optional = None,
                        return_fig_ax=False):
    """This function draws LAFs, tentative matches, inliers epipolar lines (if F is provided),
    and image1 corners reprojection into image 2 (if H is provided)

    Args:
        lafs1: LAFs of image 1, shape :math:`(1, N1, 2, 3)`.
        lafs2: LAFs of image 2, shape :math:`(1, N2, 2, 3)`.
        tent_idxs: Tentative match indexes (query, train) of shape :math:`(M, 2)`.
        img1: First image, path/tensor/numpy array.
        img2: Second image, path/tensor/numpy array.
        inlier_mask: Optional boolean array/list of length M marking inliers
            among ``tent_idxs``.
        draw_dict: Drawing options. Keys: ``inlier_color`` (RGB tuple or
            None to skip drawing inliers), ``tentative_color`` (RGB tuple or
            None to skip drawing tentative matches), ``feature_color`` (RGB
            tuple or None to skip drawing all detected LAFs), ``vertical``
            (bool, stack images vertically instead of horizontally).
        Fm: Optional fundamental matrix of shape :math:`(3, 3)` used to draw
            inlier epipolar lines.
        H: Optional homography of shape :math:`(3, 3)` used to draw the
            reprojection of image 1's corners into image 2.
        fig: Optional existing matplotlib figure to draw on.
        ax: Optional existing matplotlib axes to draw on.
        return_fig_ax: If True, return the figure and axes instead of None.

    Returns:
        Tuple of ``(fig, ax)`` if ``return_fig_ax`` is True, otherwise None.

    Example:
        >>> match_dists, match_idxs = kornia.feature.match_snn(descs1, descs2, 0.98)
        >>> draw_LAF_matches(lafs1, lafs2, match_idxs, img1, img2, mask,
        ...     draw_dict={"inlier_color": (0.2, 1, 0.2),
        ...                "tentative_color": (0.8, 0.8, 0),
        ...                "feature_color": None, "vertical": False}, H=H)
    """
    from kornia_moons.feature import to_numpy_image, to_np, to_torch
    if inlier_mask is not None:
        inlier_mask = np.array(inlier_mask).reshape(-1)
    lafs1 = to_torch(lafs1).detach().cpu().float()
    lafs2 = to_torch(lafs2).detach().cpu().float()
    tent_idxs = to_torch(tent_idxs).detach().cpu().long()
    img1 = to_numpy_image(img1)
    img2 = to_numpy_image(img2)
    img1, img2 = _promote_to_matching_channels(img1, img2)

    h,w = img1.shape[:2]
    h2,w2 = img2.shape[:2]

    corners_pts = np.float32([[0, 0],
                              [0, h-1],
                              [w-1, h-1],
                              [w-1, 0],
                              [0, 0] ]).reshape(-1,1,2)


    xy1 = KF.get_laf_center(lafs1).reshape(-1, 2)
    xy2 = KF.get_laf_center(lafs2).reshape(-1, 2)
    # If we have no axes, create one
    if (fig is None and ax is None):
        fig, ax = plt.subplots(1,1, figsize=(20,10))
    if (fig is not None and ax is None):
        ax = fig.add_axes([0, 0, 1, 1])

    tent_corrs = torch.stack([xy1[tent_idxs[:,0]], xy2[tent_idxs[:,1]]])
    try:
        vert = draw_dict['vertical']
    except:
        vert = False
    if vert:
        tent_corrs[1,:,1]+=h # shift for the 2nd image
    else:
        tent_corrs[1,:,0]+=w # shift for the 2nd image
    if H is not None:
        dst_corners = cv2.perspectiveTransform(corners_pts, H)
        if vert:
            dst_corners[...,1]+=h
        else:
            dst_corners[...,0]+=w
    # Prepraring canvas
    if not vert:
        if len(img1.shape) == 3:
            new_shape = (max(h, h2), w + w2, img1.shape[2])
        elif len(img1.shape) == 2:
            new_shape = (max(h, h2), w + w2)
    else:
        if len(img1.shape) == 3:
            new_shape = (h + h2,  max(w, w2), img1.shape[2])
        elif len(img1.shape) == 2:
            new_shape = (h + h2,  max(w, w2))        
    new_img = np.zeros(new_shape, type(img1.flat[0]))  
    # Place images onto the new image.
    if not vert:
        new_img[0:h, 0:w] = img1
        new_img[0:h2, w:w + w2] = img2
    else:
        new_img[0:h, 0:w] = img1
        new_img[h:h+h2, 0:w2] = img2

    x1, y1 = to_np(KF.laf.get_laf_pts_to_draw(lafs1, 0))
    x2, y2 = to_np(KF.laf.get_laf_pts_to_draw(lafs2, 0))
    if vert:
        y2+=h
    else:
        x2+=w
    # Drawing features
    try:
        fc = draw_dict['feature_color']
    except:
        fc = None
    if fc is not None:
        ax.plot(x1, y1, color=fc)
        ax.plot(x2, y2, color=fc)

    tent_corrs = tent_corrs.detach().cpu().numpy()
    #Drawing tentatives
    try:
        tc = draw_dict['tentative_color']
    except:
        tc = None
    if tc is not None:
        ax.plot(tent_corrs[...,0], tent_corrs[...,1], color=tc)
        ax.plot(x1[:, tent_idxs[:,0]], y1[:, tent_idxs[:,0]], color=tc)
        ax.plot(x2[:, tent_idxs[:,1]], y2[:, tent_idxs[:,1]], color=tc)
    try:
        ic = draw_dict['inlier_color']
    except:
        ic = None
    if (ic is not None) and (inlier_mask is not None):
        inlier_mask = inlier_mask > 0
        ax.plot(tent_corrs[..., inlier_mask, 0], tent_corrs[...,inlier_mask, 1], color=ic)
        ax.plot(x1[:, tent_idxs[inlier_mask,0]], y1[:, tent_idxs[inlier_mask,0]], color=ic)
        ax.plot(x2[:, tent_idxs[inlier_mask,1]], y2[:, tent_idxs[inlier_mask,1]], color=ic)
    if H is not None:
        ax.plot(corners_pts[:,0,0], corners_pts[:,0,1], color=(0,0,1))
        ax.plot(dst_corners[:,0,0], dst_corners[:,0,1], color=(0,0,1))
    if (Fm is not None):
        if inlier_mask is None:
            inlier_mask = [True for i in range(len(tent_idxs))]
        tent_corrs2 = torch.stack([xy1[tent_idxs[:,0]], xy2[tent_idxs[:,1]]])
        inl1 = tent_corrs2[0, inlier_mask]
        inl2 = tent_corrs2[1, inlier_mask]
        Ff = torch.from_numpy(Fm).float()
        epi1 = K.geometry.compute_correspond_epilines(inl2.reshape(1,-1, 2), Ff.t()[None])[0]
        epiline1 = epilines_to_start_end_points(epi1, h, w)

        epi2 = K.geometry.compute_correspond_epilines(inl1.reshape(1,-1, 2), Ff[None])[0]
        epiline2 = epilines_to_start_end_points(epi2, h2, w2)
        if vert:
            epiline2[...,1]+=h # shift for the 2nd image
        else:
            epiline2[...,0]+=w # shift for the 2nd image            

        ax.plot(epiline1[:,:,0], epiline1[:,:,1], color='purple')
        ax.plot(epiline2[:,:,0], epiline2[:,:,1], color='purple')
    # Finally clip the image
    ax.imshow(new_img)
    if not vert:
        ax.set_xlim([0,w+w2])
        ax.set_ylim([max(h,h2),0])
        ax.margins(0,0)
    else:
        ax.set_xlim([0,max(w,w2)])
        ax.set_ylim([h+h2, 0])
        ax.margins(0,0)
    if return_fig_ax : return fig, ax
    return

draw_point_matches(pts1, pts2, img1, img2, inlier_mask=None, draw_dict={'inlier_color': (0.2, 1, 0.2), 'tentative_color': (0.8, 0.8, 0), 'vertical': False}, Fm=None, H=None, fig=None, ax=None, return_fig_ax=False)

Draw already-corresponded point matches, as produced by detector-free matchers such as LoFTR, LightGlue, or DISK.

Point i of pts1 is matched to point i of pts2; the points are wrapped into unit-scale upright LAFs and drawn with :func:draw_LAF_matches.

Parameters:

Name Type Description Default
pts1

Matched points in image 1, array/tensor of shape :math:(N, 2).

 required
pts2

Matched points in image 2, array/tensor of shape :math:(N, 2).

 required
img1

First image, path/tensor/numpy array.

 required
img2

Second image, path/tensor/numpy array.

 required
inlier_mask

Optional boolean array/list of length N marking inliers.

 None
draw_dict

Drawing options, see :func:draw_LAF_matches.

 {'inlier_color': (0.2, 1, 0.2), 'tentative_color': (0.8, 0.8, 0), 'vertical': False}
Fm Optional[array]

Optional fundamental matrix of shape :math:(3, 3) used to draw inlier epipolar lines.

 None
H Optional[array]

Optional homography of shape :math:(3, 3) used to draw the reprojection of image 1's corners into image 2.

 None
fig

Optional existing matplotlib figure to draw on.

 None
ax Optional

Optional existing matplotlib axes to draw on.

 None
return_fig_ax

If True, return the figure and axes instead of None.

 False

Returns:

Type Description

Tuple of (fig, ax) if return_fig_ax is True, otherwise None.
Example

out = KF.LoFTR(pretrained='outdoor')({"image0": timg1, "image1": timg2}) mkpts0 = out['keypoints0'].cpu().numpy() mkpts1 = out['keypoints1'].cpu().numpy() Fm, inliers = cv2.findFundamentalMat(mkpts0, mkpts1, cv2.USAC_MAGSAC, 0.5) draw_point_matches(mkpts0, mkpts1, img1, img2, inliers, Fm=Fm)

Source code in kornia_moons/viz.py 
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
def draw_point_matches(pts1, pts2, img1, img2, inlier_mask=None,
                       draw_dict={"inlier_color": (0.2, 1, 0.2),
                                  "tentative_color": (0.8, 0.8, 0),
                                  "vertical": False},
                       Fm: Optional[np.array] = None, H: Optional[np.array] = None,
                       fig=None, ax: Optional = None,
                       return_fig_ax=False):
    """Draw already-corresponded point matches, as produced by detector-free
    matchers such as LoFTR, LightGlue, or DISK.

    Point ``i`` of ``pts1`` is matched to point ``i`` of ``pts2``; the points
    are wrapped into unit-scale upright LAFs and drawn with
    :func:`draw_LAF_matches`.

    Args:
        pts1: Matched points in image 1, array/tensor of shape :math:`(N, 2)`.
        pts2: Matched points in image 2, array/tensor of shape :math:`(N, 2)`.
        img1: First image, path/tensor/numpy array.
        img2: Second image, path/tensor/numpy array.
        inlier_mask: Optional boolean array/list of length N marking inliers.
        draw_dict: Drawing options, see :func:`draw_LAF_matches`.
        Fm: Optional fundamental matrix of shape :math:`(3, 3)` used to draw
            inlier epipolar lines.
        H: Optional homography of shape :math:`(3, 3)` used to draw the
            reprojection of image 1's corners into image 2.
        fig: Optional existing matplotlib figure to draw on.
        ax: Optional existing matplotlib axes to draw on.
        return_fig_ax: If True, return the figure and axes instead of None.

    Returns:
        Tuple of ``(fig, ax)`` if ``return_fig_ax`` is True, otherwise None.

    Example:
        >>> out = KF.LoFTR(pretrained='outdoor')({"image0": timg1, "image1": timg2})
        >>> mkpts0 = out['keypoints0'].cpu().numpy()
        >>> mkpts1 = out['keypoints1'].cpu().numpy()
        >>> Fm, inliers = cv2.findFundamentalMat(mkpts0, mkpts1, cv2.USAC_MAGSAC, 0.5)
        >>> draw_point_matches(mkpts0, mkpts1, img1, img2, inliers, Fm=Fm)
    """
    from kornia_moons.feature import to_torch
    pts1 = to_torch(pts1).detach().cpu().float().reshape(-1, 2)
    pts2 = to_torch(pts2).detach().cpu().float().reshape(-1, 2)
    if len(pts1) != len(pts2):
        raise ValueError(
            f'pts1 and pts2 must be matched 1:1, got {len(pts1)} and {len(pts2)} points')
    lafs = []
    for pts in (pts1, pts2):
        n = pts.shape[0]
        scale = torch.ones(1, n, 1, 1)
        ori = torch.zeros(1, n, 1)
        lafs.append(KF.laf_from_center_scale_ori(pts[None], scale, ori))
    tent_idxs = torch.arange(len(pts1)).view(-1, 1).repeat(1, 2)
    return draw_LAF_matches(lafs[0], lafs[1], tent_idxs, img1, img2, inlier_mask,
                            draw_dict, Fm, H, fig, ax, return_fig_ax)

draw_LAF_matches_from_result_dict(result_dict, img1, img2, draw_dict={'inlier_color': (0.2, 1, 0.2), 'tentative_color': (0.8, 0.8, 0), 'feature_color': (0.2, 0.5, 1), 'vertical': False})

Draw matches from a result dictionary produced by a matching pipeline.

Parameters:

Name Type Description Default
result_dict

Dictionary with keys feat_dict1 and feat_dict2 (each {'lafs': tensor of shape (1, N1, 2, 3)} and (1, N2, 2, 3) respectively), tents_dict ({'idxs': long tensor of shape (M, 2)} with tentative match indexes), inlier_mask (list/array of M bools), and optionally 'H' (homography, :math:(3, 3) np.array) and/or 'F' (fundamental matrix, :math:(3, 3) np.array).

 required
img1

First image, path/tensor/numpy array.

 required
img2

Second image, path/tensor/numpy array.

 required
draw_dict

Drawing options, see :func:draw_LAF_matches.

 {'inlier_color': (0.2, 1, 0.2), 'tentative_color': (0.8, 0.8, 0), 'feature_color': (0.2, 0.5, 1), 'vertical': False}

Returns:

Type Description

None. Forwards to :func:draw_LAF_matches without return_fig_ax.
Source code in kornia_moons/viz.py 
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
def draw_LAF_matches_from_result_dict(result_dict,
                        img1, img2, 
                        draw_dict={"inlier_color": (0.2, 1, 0.2),
                               "tentative_color": (0.8, 0.8, 0), 
                               "feature_color": (0.2, 0.5, 1),
                                  "vertical": False}):
    """Draw matches from a result dictionary produced by a matching pipeline.

    Args:
        result_dict: Dictionary with keys ``feat_dict1`` and ``feat_dict2``
            (each ``{'lafs': tensor of shape (1, N1, 2, 3)}`` and ``(1, N2, 2, 3)`` respectively), ``tents_dict``
            (``{'idxs': long tensor of shape (M, 2)}`` with tentative match
            indexes), ``inlier_mask`` (list/array of M bools), and optionally
            ``'H'`` (homography, :math:`(3, 3)` np.array) and/or ``'F'``
            (fundamental matrix, :math:`(3, 3)` np.array).
        img1: First image, path/tensor/numpy array.
        img2: Second image, path/tensor/numpy array.
        draw_dict: Drawing options, see :func:`draw_LAF_matches`.

    Returns:
        None. Forwards to :func:`draw_LAF_matches` without ``return_fig_ax``.
    """
    if 'H' in result_dict:
        H = result_dict['H']
    else:
        H = None
    if 'F' in result_dict:
        Fm = result_dict['F']
    else:
        Fm = None
    return draw_LAF_matches(result_dict['feat_dict1']['lafs'], 
                        result_dict['feat_dict2']['lafs'],
                               result_dict['tents_dict']['idxs'],  
                        img1, img2, result_dict['inlier_mask'], 
                        draw_dict, Fm, H)

draw_LAF_inliers_perspective_repjojected(lafs1, lafs2, tent_idxs, img1, img2, inlier_mask=None, draw_dict={'inlier_color': (0.2, 1, 0.2), 'reprojected_color': (0.2, 0.5, 1), 'vertical': False}, H=None, fig=None, ax=None, return_fig_ax=False)

This function draws tentative matches and inliers given the homography H

Note that the function name keeps its original typo (repjojected) for backward compatibility.

Parameters:

Name Type Description Default
lafs1

LAFs of image 1, shape :math:(1, N, 2, 3).

 required
lafs2

LAFs of image 2, shape :math:(1, N, 2, 3).

 required
tent_idxs

Tentative match indexes (query, train) of shape :math:(M, 2).

 required
img1

First image, path/tensor/numpy array.

 required
img2

Second image, path/tensor/numpy array.

 required
inlier_mask

Boolean array/list of length M marking inliers among tent_idxs.

 None
draw_dict

Drawing options. Keys: inlier_color (RGB tuple or None to skip drawing inliers), reprojected_color (RGB tuple or None to skip drawing the reprojected LAFs), vertical (bool, stack images vertically instead of horizontally).

 {'inlier_color': (0.2, 1, 0.2), 'reprojected_color': (0.2, 0.5, 1), 'vertical': False}
H array

Homography of shape :math:(3, 3) used to reproject LAFs between the two images.

 None
fig

Optional existing matplotlib figure to draw on.

 None
ax Optional

Optional existing matplotlib axes to draw on.

 None
return_fig_ax

If True, return the figure and axes instead of None.

 False

Returns:

Type Description

Tuple of (fig, ax) if return_fig_ax is True, otherwise None.
Source code in kornia_moons/viz.py 
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
def draw_LAF_inliers_perspective_repjojected(lafs1, lafs2, tent_idxs,  
                        img1, img2, inlier_mask = None, 
                        draw_dict={"inlier_color": (0.2, 1, 0.2),
                               "reprojected_color": (0.2, 0.5, 1),
                                "vertical": False}, 
                        H: np.array = None,
                        fig = None, ax: Optional = None,
                        return_fig_ax=False):
    """This function draws tentative matches and inliers given the homography H

    Note that the function name keeps its original typo
    (``repjojected``) for backward compatibility.

    Args:
        lafs1: LAFs of image 1, shape :math:`(1, N, 2, 3)`.
        lafs2: LAFs of image 2, shape :math:`(1, N, 2, 3)`.
        tent_idxs: Tentative match indexes (query, train) of shape :math:`(M, 2)`.
        img1: First image, path/tensor/numpy array.
        img2: Second image, path/tensor/numpy array.
        inlier_mask: Boolean array/list of length M marking inliers among
            ``tent_idxs``.
        draw_dict: Drawing options. Keys: ``inlier_color`` (RGB tuple or
            None to skip drawing inliers), ``reprojected_color`` (RGB tuple
            or None to skip drawing the reprojected LAFs), ``vertical``
            (bool, stack images vertically instead of horizontally).
        H: Homography of shape :math:`(3, 3)` used to reproject LAFs between
            the two images.
        fig: Optional existing matplotlib figure to draw on.
        ax: Optional existing matplotlib axes to draw on.
        return_fig_ax: If True, return the figure and axes instead of None.

    Returns:
        Tuple of ``(fig, ax)`` if ``return_fig_ax`` is True, otherwise None.
    """
    import kornia as K
    import kornia.feature as KF
    import kornia.geometry as KG
    from kornia_moons.feature import to_numpy_image, to_np, to_torch
    inlier_mask = np.array(inlier_mask).reshape(-1)
    lafs1 = to_torch(lafs1).detach().cpu().float()
    lafs2 = to_torch(lafs2).detach().cpu().float()
    tent_idxs = to_torch(tent_idxs).detach().cpu().long()
    img1 = to_numpy_image(img1)
    img2 = to_numpy_image(img2)
    img1, img2 = _promote_to_matching_channels(img1, img2)

    h,w = img1.shape[:2]
    h2,w2 = img2.shape[:2]

    lafs1_in2 = KF.perspective_transform_lafs(torch.from_numpy(H).float()[None],
                                              lafs1)
    lafs2_in1 = KF.perspective_transform_lafs(torch.inverse(torch.from_numpy(H).float()[None]),
                                              lafs2)
    xy1 = KF.get_laf_center(lafs1).reshape(-1, 2)
    xy2 = KF.get_laf_center(lafs2).reshape(-1, 2)

    # If we have no axes, create one
    if (fig is None and ax is None):
        fig, ax = plt.subplots(1,1, figsize=(20,10))
    if (fig is not None and ax is None):
        ax = fig.add_axes([0, 0, 1, 1])

    tent_corrs_in1 = torch.stack([xy1[tent_idxs[:,0]],
                                  KF.get_laf_center(lafs2_in1).reshape(-1, 2)[tent_idxs[:,1]]])

    tent_corrs_in2 = torch.stack([KF.get_laf_center(lafs1_in2).reshape(-1, 2)[tent_idxs[:,0]],
                                  xy2[tent_idxs[:,1]]])

    try:
        vert = draw_dict['vertical']
    except:
        vert = False
    if vert:
        tent_corrs_in2[:,:,1]+=h # shift for the 2nd image
    else:
        tent_corrs_in2[:,:,0]+=w # shift for the 2nd image
    # Prepraring canvas
    if not vert:
        if len(img1.shape) == 3:
            new_shape = (max(h, h2), w + w2, img1.shape[2])
        elif len(img1.shape) == 2:
            new_shape = (max(h, h2), w + w2)
    else:
        if len(img1.shape) == 3:
            new_shape = (h + h2,  max(w, w2), img1.shape[2])
        elif len(img1.shape) == 2:
            new_shape = (h + h2,  max(w, w2))        
    new_img = np.zeros(new_shape, type(img1.flat[0]))  
    # Place images onto the new image.
    if not vert:
        new_img[0:h, 0:w] = img1
        new_img[0:h2, w:w + w2] = img2
    else:
        new_img[0:h, 0:w] = img1
        new_img[h:h+h2, 0:w2] = img2

    x1,y1 = to_np(KF.laf.get_laf_pts_to_draw(lafs1, 0))
    x2,y2 = to_np(KF.laf.get_laf_pts_to_draw(lafs2, 0))

    x1in2, y1in2 = to_np(KF.laf.get_laf_pts_to_draw(lafs1_in2, 0))
    x2in1, y2in1 = to_np(KF.laf.get_laf_pts_to_draw(lafs2_in1, 0))

    if vert:
        y2+=h
        y1in2+=h
    else:
        x2+=w
        x1in2+=w

    try:
        ic = draw_dict['inlier_color']
    except:
        ic = None
    if (ic is not None) and (inlier_mask is not None):
        inlier_mask = inlier_mask > 0
        ax.plot(tent_corrs_in1[..., inlier_mask, 0], tent_corrs_in1[...,inlier_mask, 1], color=ic)#, markersize=15, marker='x')
        ax.plot(tent_corrs_in2[..., inlier_mask, 0],
                tent_corrs_in2[...,inlier_mask, 1], color=ic)#, markersize=15, marker='x')
        ax.plot(x1[:, tent_idxs[inlier_mask,0]], y1[:, tent_idxs[inlier_mask,0]], color=ic)#, markersize=15)
        ax.plot(x2[:, tent_idxs[inlier_mask,1]], y2[:, tent_idxs[inlier_mask,1]], color=ic)# markersize=15)
    try:
        rc = draw_dict['reprojected_color']
    except:
        rc = None
    if rc is not None:
        inlier_mask = inlier_mask > 0
        ax.plot(x1in2[:, tent_idxs[inlier_mask,0]], y1in2[:, tent_idxs[inlier_mask,0]], color=rc)#, markersize=15)
        ax.plot(x2in1[:, tent_idxs[inlier_mask,1]], y2in1[:, tent_idxs[inlier_mask,1]], color=rc)#, markersize=15)

    # Finally clip the image
    ax.imshow(new_img)
    if not vert:
        ax.set_xlim([0,w+w2])
        ax.set_ylim([max(h,h2),0])
        ax.margins(0,0)
    else:
        ax.set_xlim([0,max(w,w2)])
        ax.set_ylim([h+h2, 0])
        ax.margins(0,0)
    if return_fig_ax : return fig, ax
    return

draw_epipolar_errors_in_single_image(kp1, kp2, Fm1to2, img, draw_dict={'error_color': (1, 0.2, 0.2), 'feature_color': (0.2, 0.5, 1), 'figsize': (10, 10), 'markersize': 8}, img_index=2, ax=None, title=None)

This function draws epipolar errors in single image

Parameters:

Name Type Description Default
kp1 array

Keypoints in image 1, array of shape :math:(N, 2).

 required
kp2 array

Keypoints in image 2, array of shape :math:(N, 2).

 required
Fm1to2 array

Fundamental matrix mapping points from image 1 to image 2, of shape :math:(3, 3).

 required
img

Image to draw on (the one indexed by img_index), path, tensor, or numpy array.

 required
draw_dict

Drawing options. Keys: error_color (RGB tuple for the error lines and estimated projections), feature_color (RGB tuple for the ground-truth points), figsize (figure size used when ax is None), markersize (marker size for the scatter points).

 {'error_color': (1, 0.2, 0.2), 'feature_color': (0.2, 0.5, 1), 'figsize': (10, 10), 'markersize': 8}
img_index int

Which image (1 or 2) kp1/kp2/img refer to; when 1, kp1/kp2 and Fm1to2 are swapped/transposed accordingly.

 2
ax Optional

Optional existing matplotlib axes to draw on.

 None
title

Optional plot title.

 None

Returns:

Type Description

The matplotlib axes the errors were drawn on.
Source code in kornia_moons/viz.py 
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
def draw_epipolar_errors_in_single_image(kp1: np.array, kp2: np.array,
                                         Fm1to2: np.array, img,
                                          draw_dict={"error_color": (1, 0.2, 0.2),
                                                     "feature_color": (0.2, 0.5, 1),
                                                     "figsize": (10,10),
                                                     "markersize": 8},
                                         img_index: int = 2,
                                         ax: Optional = None,
                                         title = None):
    """This function draws epipolar errors in single image

    Args:
        kp1: Keypoints in image 1, array of shape :math:`(N, 2)`.
        kp2: Keypoints in image 2, array of shape :math:`(N, 2)`.
        Fm1to2: Fundamental matrix mapping points from image 1 to image 2,
            of shape :math:`(3, 3)`.
        img: Image to draw on (the one indexed by ``img_index``), path,
            tensor, or numpy array.
        draw_dict: Drawing options. Keys: ``error_color`` (RGB tuple for the
            error lines and estimated projections), ``feature_color`` (RGB
            tuple for the ground-truth points), ``figsize`` (figure size
            used when ``ax`` is None), ``markersize`` (marker size for the
            scatter points).
        img_index: Which image (1 or 2) ``kp1``/``kp2``/``img`` refer to;
            when 1, ``kp1``/``kp2`` and ``Fm1to2`` are swapped/transposed
            accordingly.
        ax: Optional existing matplotlib axes to draw on.
        title: Optional plot title.

    Returns:
        The matplotlib axes the errors were drawn on.
    """
    from kornia.geometry.epipolar import get_closest_point_on_epipolar_line
    from kornia_moons.feature import to_numpy_image, cv2_matches_from_kornia, to_np, to_torch

    img = to_numpy_image(img)
    pts1 = to_torch(kp1)[None]
    pts2 = to_torch(kp2)[None]
    Fm = to_torch(Fm1to2)
    if len(Fm.shape) == 2:
        Fm = Fm[None]
    assert img_index in [1,2]
    if img_index == 1:
        pts1, pts2 = pts2, pts1
        Fm = Fm.transpose(1, 2)
    closest_pts = get_closest_point_on_epipolar_line(pts1, pts2, Fm)

    # If we have no axes, create one
    if ax is None:
        fig, ax = plt.subplots(figsize=draw_dict["figsize"])
    ax.imshow(img)
    plt.scatter(pts2[0, :, 0].numpy(),
                pts2[0, :, 1].numpy(),
                s=draw_dict["markersize"]**2,
                label='GT points',
                color = draw_dict['feature_color'])
    plt.scatter(closest_pts[0, :, 0].numpy(),
                closest_pts[0, :, 1].numpy(),
                s=draw_dict["markersize"]**2,
                label='est. projection',
                color = draw_dict['error_color'], marker='x')

    plt.plot(torch.cat([pts2[0, :, 0].view(1,-1),
                        closest_pts[0, :, 0].view(1,-1)],dim=0).numpy(),
             torch.cat([pts2[0, :, 1].view(1,-1),
                        closest_pts[0, :, 1].view(1,-1)], dim=0).numpy(),
             label='error', color=draw_dict['error_color'])

    plt.legend(['GT points','est. projection','errors'])
    if title is not None:
        plt.title(title)
    return ax

plot_images(imgs, titles=None, cmaps='gray', dpi=100, size=6, pad=0.5)

Plot a horizontal strip of images and establish the figure that :func:plot_lines and :func:plot_color_line_matches draw on.

Parameters:

Name Type Description Default
imgs

List of NumPy or PyTorch images, RGB :math:(H, W, 3) or mono :math:(H, W).

 required
titles

Optional list of strings, as titles for each image.

 None
cmaps

Colormap, or list of colormaps (one per image), used for monochrome images.

 'gray'
dpi

Figure resolution in dots per inch.

 100
size

Width in inches allotted per image; the figure height is 3/4 of it.

 6
pad

Padding passed to fig.tight_layout.

 0.5

Returns:

Type Description

None. The created figure and axes become the current matplotlib

figure, ready for :func:plot_lines to draw on.
Source code in kornia_moons/viz.py 
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
def plot_images(imgs, titles=None, cmaps="gray", dpi=100, size=6, pad=0.5):
    """Plot a horizontal strip of images and establish the figure that
    :func:`plot_lines` and :func:`plot_color_line_matches` draw on.

    Args:
        imgs: List of NumPy or PyTorch images, RGB :math:`(H, W, 3)` or
            mono :math:`(H, W)`.
        titles: Optional list of strings, as titles for each image.
        cmaps: Colormap, or list of colormaps (one per image), used for
            monochrome images.
        dpi: Figure resolution in dots per inch.
        size: Width in inches allotted per image; the figure height is ``3/4`` of it.
        pad: Padding passed to ``fig.tight_layout``.

    Returns:
        None. The created figure and axes become the current matplotlib
        figure, ready for :func:`plot_lines` to draw on.
    """
    import matplotlib
    import matplotlib.colors as mcolors
    import matplotlib.pyplot as plt

    n = len(imgs)
    if not isinstance(cmaps, (list, tuple)):
        cmaps = [cmaps] * n
    figsize = (size * n, size * 3 / 4) if size is not None else None
    fig, ax = plt.subplots(1, n, figsize=figsize, dpi=dpi)
    if n == 1:
        ax = [ax]
    for i in range(n):
        ax[i].imshow(imgs[i], cmap=plt.get_cmap(cmaps[i]))
        ax[i].get_yaxis().set_ticks([])
        ax[i].get_xaxis().set_ticks([])
        ax[i].set_axis_off()
        for spine in ax[i].spines.values():  # remove frame
            spine.set_visible(False)
        if titles:
            ax[i].set_title(titles[i])
    fig.tight_layout(pad=pad)

plot_lines(lines, line_colors='orange', point_colors='cyan', ps=4, lw=2, indices=(0, 1))

Plot line segments and their endpoints on the current figure's axes.

Must be called after :func:plot_images, which creates the figure and axes this function draws on.

Parameters:

Name Type Description Default
lines

List of arrays of shape :math:(N, 2, 2), one per image, with [..., 0] the y coordinate and [..., 1] the x coordinate of each endpoint.

 required
line_colors

Color, or list of colors (one per image), for the line segments.

 'orange'
point_colors

Color, or list of colors (one per image), for the endpoints.

 'cyan'
ps

Size of the endpoint markers, in points.

 4
lw

Line width, in points.

 2
indices

Indexes of the current figure's axes to draw the lines on.

 (0, 1)

Returns:

Type Description

None. The lines and points are added to the existing axes in place.
Source code in kornia_moons/viz.py 
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
def plot_lines(lines, line_colors="orange", point_colors="cyan", ps=4, lw=2, indices=(0, 1)):
    """Plot line segments and their endpoints on the current figure's axes.

    Must be called after :func:`plot_images`, which creates the figure and
    axes this function draws on.

    Args:
        lines: List of arrays of shape :math:`(N, 2, 2)`, one per image,
            with ``[..., 0]`` the y coordinate and ``[..., 1]`` the x
            coordinate of each endpoint.
        line_colors: Color, or list of colors (one per image), for the
            line segments.
        point_colors: Color, or list of colors (one per image), for the
            endpoints.
        ps: Size of the endpoint markers, in points.
        lw: Line width, in points.
        indices: Indexes of the current figure's axes to draw the lines on.

    Returns:
        None. The lines and points are added to the existing axes in place.
    """
    import matplotlib
    import matplotlib.colors as mcolors
    import matplotlib.pyplot as plt
    if not isinstance(line_colors, list):
        line_colors = [line_colors] * len(lines)
    if not isinstance(point_colors, list):
        point_colors = [point_colors] * len(lines)

    fig = plt.gcf()
    ax = fig.axes
    assert len(ax) > max(indices)
    axes = [ax[i] for i in indices]
    fig.canvas.draw()

    # Plot the lines and junctions
    for a, l, lc, pc in zip(axes, lines, line_colors, point_colors):
        for i in range(len(l)):
            line = matplotlib.lines.Line2D(
                (l[i, 1, 1], l[i, 0, 1]),
                (l[i, 1, 0], l[i, 0, 0]),
                zorder=1,
                c=lc,
                linewidth=lw,
            )
            a.add_line(line)
        pts = l.reshape(-1, 2)
        a.scatter(pts[:, 1], pts[:, 0], c=pc, s=ps, linewidths=0, zorder=2)

plot_color_line_matches(lines, lw=2, indices=(0, 1))

Plot matched line segments on the current figure's axes, giving each match its own color.

Must be called after :func:plot_images, which creates the figure and axes this function draws on. Used to visualize SOLD2-style line matches between two images.

Parameters:

Name Type Description Default
lines

List of arrays of shape :math:(N, 2, 2), one per image, with [..., 0] the y coordinate and [..., 1] the x coordinate of each endpoint. Line i in each image is drawn with the same random color, so index i must denote the same match across images.

 required
lw

Line width, in points.

 2
indices

Indexes of the current figure's axes to draw the lines on.

 (0, 1)

Returns:

Type Description

None. The lines are added to the existing axes in place.
Source code in kornia_moons/viz.py 
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
def plot_color_line_matches(lines, lw=2, indices=(0, 1)):
    """Plot matched line segments on the current figure's axes, giving each
    match its own color.

    Must be called after :func:`plot_images`, which creates the figure and
    axes this function draws on. Used to visualize SOLD2-style line matches
    between two images.

    Args:
        lines: List of arrays of shape :math:`(N, 2, 2)`, one per image,
            with ``[..., 0]`` the y coordinate and ``[..., 1]`` the x
            coordinate of each endpoint. Line ``i`` in each image is drawn
            with the same random color, so index ``i`` must denote the same
            match across images.
        lw: Line width, in points.
        indices: Indexes of the current figure's axes to draw the lines on.

    Returns:
        None. The lines are added to the existing axes in place.
    """
    import matplotlib
    import matplotlib.colors as mcolors
    import matplotlib.pyplot as plt

    n_lines = len(lines[0])

    cmap = plt.get_cmap("nipy_spectral", lut=n_lines)
    colors = np.array([mcolors.rgb2hex(cmap(i)) for i in range(cmap.N)])

    np.random.shuffle(colors)

    fig = plt.gcf()
    ax = fig.axes
    assert len(ax) > max(indices)
    axes = [ax[i] for i in indices]
    fig.canvas.draw()

    # Plot the lines
    for a, l in zip(axes, lines):
        for i in range(len(l)):
            line = matplotlib.lines.Line2D(
                (l[i, 1, 1], l[i, 0, 1]),
                (l[i, 1, 0], l[i, 0, 0]),
                zorder=1,
                c=colors[i],
                linewidth=lw,
            )
            a.add_line(line)