์ฝ˜ํ…์ธ ๋กœ ๊ฑด๋„ˆ๋›ฐ๊ธฐ

์ฐธ์กฐ ultralytics/trackers/utils/matching.py

์ฐธ๊ณ 

์ด ํŒŒ์ผ์€ https://github.com/ultralytics/ ultralytics/blob/main/ ultralytics/trackers/utils/matching .py์—์„œ ํ™•์ธํ•  ์ˆ˜ ์žˆ์Šต๋‹ˆ๋‹ค. ๋ฌธ์ œ๋ฅผ ๋ฐœ๊ฒฌํ•˜๋ฉด ํ’€ ๋ฆฌํ€˜์ŠคํŠธ ๐Ÿ› ๏ธ ์— ๊ธฐ์—ฌํ•˜์—ฌ ๋ฌธ์ œ๋ฅผ ํ•ด๊ฒฐํ•˜๋„๋ก ๋„์™€์ฃผ์„ธ์š”. ๊ฐ์‚ฌํ•ฉ๋‹ˆ๋‹ค ๐Ÿ™!



ultralytics.trackers.utils.matching.linear_assignment(cost_matrix, thresh, use_lap=True)

scipy ๋˜๋Š” lap.lapjv๋ฅผ ์‚ฌ์šฉํ•˜์—ฌ ์„ ํ˜• ํ• ๋‹น์„ ์ˆ˜ํ–‰ํ•ฉ๋‹ˆ๋‹ค.

๋งค๊ฐœ๋ณ€์ˆ˜:

์ด๋ฆ„ ์œ ํ˜• ์„ค๋ช… ๊ธฐ๋ณธ๊ฐ’
cost_matrix ndarray

๊ณผ์ œ์˜ ๋น„์šฉ ๊ฐ’์ด ํฌํ•จ๋œ ํ–‰๋ ฌ์ž…๋‹ˆ๋‹ค.

ํ•„์ˆ˜
thresh float

๊ณผ์ œ๋ฅผ ์œ ํšจํ•œ ๊ฒƒ์œผ๋กœ ๊ฐ„์ฃผํ•˜๋Š” ์ž„๊ณ„๊ฐ’์ž…๋‹ˆ๋‹ค.

ํ•„์ˆ˜
use_lap bool

lap.lapjv ์‚ฌ์šฉ ์—ฌ๋ถ€์ž…๋‹ˆ๋‹ค. ๊ธฐ๋ณธ๊ฐ’์€ True์ž…๋‹ˆ๋‹ค.

True

๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค:

์œ ํ˜• ์„ค๋ช…
tuple

ํŠœํ”Œ: - ์ผ์น˜ํ•˜๋Š” ์ธ๋ฑ์Šค - 'A'์—์„œ ์ผ์น˜ํ•˜์ง€ ์•Š๋Š” ์ธ๋ฑ์Šค - 'B'์—์„œ ์ผ์น˜ํ•˜์ง€ ์•Š๋Š” ์ธ๋ฑ์Šค

์˜ ์†Œ์Šค ์ฝ”๋“œ ultralytics/trackers/utils/matching.py
def linear_assignment(cost_matrix: np.ndarray, thresh: float, use_lap: bool = True) -> tuple:
    """
    Perform linear assignment using scipy or lap.lapjv.

    Args:
        cost_matrix (np.ndarray): The matrix containing cost values for assignments.
        thresh (float): Threshold for considering an assignment valid.
        use_lap (bool, optional): Whether to use lap.lapjv. Defaults to True.

    Returns:
        Tuple with:
            - matched indices
            - unmatched indices from 'a'
            - unmatched indices from 'b'
    """

    if cost_matrix.size == 0:
        return np.empty((0, 2), dtype=int), tuple(range(cost_matrix.shape[0])), tuple(range(cost_matrix.shape[1]))

    if use_lap:
        # Use lap.lapjv
        # https://github.com/gatagat/lap
        _, x, y = lap.lapjv(cost_matrix, extend_cost=True, cost_limit=thresh)
        matches = [[ix, mx] for ix, mx in enumerate(x) if mx >= 0]
        unmatched_a = np.where(x < 0)[0]
        unmatched_b = np.where(y < 0)[0]
    else:
        # Use scipy.optimize.linear_sum_assignment
        # https://docs.scipy.org/doc/scipy/reference/generated/scipy.optimize.linear_sum_assignment.html
        x, y = scipy.optimize.linear_sum_assignment(cost_matrix)  # row x, col y
        matches = np.asarray([[x[i], y[i]] for i in range(len(x)) if cost_matrix[x[i], y[i]] <= thresh])
        if len(matches) == 0:
            unmatched_a = list(np.arange(cost_matrix.shape[0]))
            unmatched_b = list(np.arange(cost_matrix.shape[1]))
        else:
            unmatched_a = list(set(np.arange(cost_matrix.shape[0])) - set(matches[:, 0]))
            unmatched_b = list(set(np.arange(cost_matrix.shape[1])) - set(matches[:, 1]))

    return matches, unmatched_a, unmatched_b



ultralytics.trackers.utils.matching.iou_distance(atracks, btracks)

ํŠธ๋ž™ ๊ฐ„ ๊ต์ฐจ์  ๊ฐ„ ๊ฒฐํ•ฉ(IoU)์„ ๊ธฐ์ค€์œผ๋กœ ๋น„์šฉ์„ ๊ณ„์‚ฐํ•ฉ๋‹ˆ๋‹ค.

๋งค๊ฐœ๋ณ€์ˆ˜:

์ด๋ฆ„ ์œ ํ˜• ์„ค๋ช… ๊ธฐ๋ณธ๊ฐ’
atracks list[STrack] | list[ndarray]

ํŠธ๋ž™ 'a' ๋˜๋Š” ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค ๋ชฉ๋ก์ž…๋‹ˆ๋‹ค.

ํ•„์ˆ˜
btracks list[STrack] | list[ndarray]

ํŠธ๋ž™ 'B' ๋˜๋Š” ๋ฐ”์šด๋”ฉ ๋ฐ•์Šค ๋ชฉ๋ก์ž…๋‹ˆ๋‹ค.

ํ•„์ˆ˜

๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค:

์œ ํ˜• ์„ค๋ช…
ndarray

IoU๋ฅผ ๊ธฐ์ค€์œผ๋กœ ๊ณ„์‚ฐ๋œ ๋น„์šฉ ํ–‰๋ ฌ์ž…๋‹ˆ๋‹ค.

์˜ ์†Œ์Šค ์ฝ”๋“œ ultralytics/trackers/utils/matching.py
def iou_distance(atracks: list, btracks: list) -> np.ndarray:
    """
    Compute cost based on Intersection over Union (IoU) between tracks.

    Args:
        atracks (list[STrack] | list[np.ndarray]): List of tracks 'a' or bounding boxes.
        btracks (list[STrack] | list[np.ndarray]): List of tracks 'b' or bounding boxes.

    Returns:
        (np.ndarray): Cost matrix computed based on IoU.
    """

    if atracks and isinstance(atracks[0], np.ndarray) or btracks and isinstance(btracks[0], np.ndarray):
        atlbrs = atracks
        btlbrs = btracks
    else:
        atlbrs = [track.xywha if track.angle is not None else track.xyxy for track in atracks]
        btlbrs = [track.xywha if track.angle is not None else track.xyxy for track in btracks]

    ious = np.zeros((len(atlbrs), len(btlbrs)), dtype=np.float32)
    if len(atlbrs) and len(btlbrs):
        if len(atlbrs[0]) == 5 and len(btlbrs[0]) == 5:
            ious = batch_probiou(
                np.ascontiguousarray(atlbrs, dtype=np.float32),
                np.ascontiguousarray(btlbrs, dtype=np.float32),
            ).numpy()
        else:
            ious = bbox_ioa(
                np.ascontiguousarray(atlbrs, dtype=np.float32),
                np.ascontiguousarray(btlbrs, dtype=np.float32),
                iou=True,
            )
    return 1 - ious  # cost matrix



ultralytics.trackers.utils.matching.embedding_distance(tracks, detections, metric='cosine')

์ž„๋ฒ ๋”ฉ์„ ๊ธฐ๋ฐ˜์œผ๋กœ ํŠธ๋ž™๊ณผ ํƒ์ง€ ๊ฐ„์˜ ๊ฑฐ๋ฆฌ๋ฅผ ๊ณ„์‚ฐํ•ฉ๋‹ˆ๋‹ค.

๋งค๊ฐœ๋ณ€์ˆ˜:

์ด๋ฆ„ ์œ ํ˜• ์„ค๋ช… ๊ธฐ๋ณธ๊ฐ’
tracks list[STrack]

ํŠธ๋ž™ ๋ชฉ๋ก.

ํ•„์ˆ˜
detections list[BaseTrack]

ํƒ์ง€ ๋ชฉ๋ก.

ํ•„์ˆ˜
metric str

๊ฑฐ๋ฆฌ ๊ณ„์‚ฐ์„ ์œ„ํ•œ ๋ฉ”ํŠธ๋ฆญ์ž…๋‹ˆ๋‹ค. ๊ธฐ๋ณธ๊ฐ’์€ '์ฝ”์‚ฌ์ธ'์ž…๋‹ˆ๋‹ค.

'cosine'

๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค:

์œ ํ˜• ์„ค๋ช…
ndarray

์ž„๋ฒ ๋”ฉ์„ ๊ธฐ๋ฐ˜์œผ๋กœ ๊ณ„์‚ฐ๋œ ๋น„์šฉ ํ–‰๋ ฌ์ž…๋‹ˆ๋‹ค.

์˜ ์†Œ์Šค ์ฝ”๋“œ ultralytics/trackers/utils/matching.py
def embedding_distance(tracks: list, detections: list, metric: str = "cosine") -> np.ndarray:
    """
    Compute distance between tracks and detections based on embeddings.

    Args:
        tracks (list[STrack]): List of tracks.
        detections (list[BaseTrack]): List of detections.
        metric (str, optional): Metric for distance computation. Defaults to 'cosine'.

    Returns:
        (np.ndarray): Cost matrix computed based on embeddings.
    """

    cost_matrix = np.zeros((len(tracks), len(detections)), dtype=np.float32)
    if cost_matrix.size == 0:
        return cost_matrix
    det_features = np.asarray([track.curr_feat for track in detections], dtype=np.float32)
    # for i, track in enumerate(tracks):
    # cost_matrix[i, :] = np.maximum(0.0, cdist(track.smooth_feat.reshape(1,-1), det_features, metric))
    track_features = np.asarray([track.smooth_feat for track in tracks], dtype=np.float32)
    cost_matrix = np.maximum(0.0, cdist(track_features, det_features, metric))  # Normalized features
    return cost_matrix



ultralytics.trackers.utils.matching.fuse_score(cost_matrix, detections)

๋น„์šฉ ํ–‰๋ ฌ๊ณผ ํƒ์ง€ ์ ์ˆ˜๋ฅผ ์œตํ•ฉํ•˜์—ฌ ๋‹จ์ผ ์œ ์‚ฌ์„ฑ ํ–‰๋ ฌ์„ ์ƒ์„ฑํ•ฉ๋‹ˆ๋‹ค.

๋งค๊ฐœ๋ณ€์ˆ˜:

์ด๋ฆ„ ์œ ํ˜• ์„ค๋ช… ๊ธฐ๋ณธ๊ฐ’
cost_matrix ndarray

๊ณผ์ œ์˜ ๋น„์šฉ ๊ฐ’์ด ํฌํ•จ๋œ ํ–‰๋ ฌ์ž…๋‹ˆ๋‹ค.

ํ•„์ˆ˜
detections list[BaseTrack]

์ ์ˆ˜๊ฐ€ ํฌํ•จ๋œ ํƒ์ง€ ๋ชฉ๋ก์ž…๋‹ˆ๋‹ค.

ํ•„์ˆ˜

๋ฐ˜ํ™˜ํ•ฉ๋‹ˆ๋‹ค:

์œ ํ˜• ์„ค๋ช…
ndarray

์œตํ•ฉ ์œ ์‚ฌ๋„ ๋งคํŠธ๋ฆญ์Šค.

์˜ ์†Œ์Šค ์ฝ”๋“œ ultralytics/trackers/utils/matching.py
def fuse_score(cost_matrix: np.ndarray, detections: list) -> np.ndarray:
    """
    Fuses cost matrix with detection scores to produce a single similarity matrix.

    Args:
        cost_matrix (np.ndarray): The matrix containing cost values for assignments.
        detections (list[BaseTrack]): List of detections with scores.

    Returns:
        (np.ndarray): Fused similarity matrix.
    """

    if cost_matrix.size == 0:
        return cost_matrix
    iou_sim = 1 - cost_matrix
    det_scores = np.array([det.score for det in detections])
    det_scores = np.expand_dims(det_scores, axis=0).repeat(cost_matrix.shape[0], axis=0)
    fuse_sim = iou_sim * det_scores
    return 1 - fuse_sim  # fuse_cost





2023-11-12 ์ƒ์„ฑ, 2023-11-25 ์—…๋ฐ์ดํŠธ๋จ
์ž‘์„ฑ์ž: glenn-jocher (3), Laughing-q (1)