Search similar object embeddings by text

Search for similar objects by text

POST/api/search/searchSimilarObjectEmbeddingsByText

Authorizations

x-auth-apikeystringheaderrequired

Your Rhombus API key. Must be accompanied by the x-auth-scheme header set to api-token (or partner-api-token for partner endpoints).

Authorizationstringheader

Instead of an API key, an OAuth access token: Bearer <token>, with the read scope.

OAuth explains tokens and scopes.

Headers

x-auth-schemeenum<string>headerdefault: api-token

Authentication scheme identifier. Use api-token or partner-api-token with the x-auth-apikey header, api or partner-api with x-auth-apikey plus a client certificate, api-oauth-token or partner-api-oauth-token with x-auth-access-token, and federated-token with x-auth-ft. A request with Authorization: Bearer <token> and no x-auth-scheme is treated as OAuth, and federated-token may also be sent as the x-auth-scheme and x-auth-ft query parameters.

Available options: api-token, api, api-oauth-token, federated-token, partner-api-token, partner-api, partner-api-oauth-token

x-auth-orgstring<RUUID>header

Client organization UUID (22-character RUUID), for partner credentials only. With it the call runs in that client organization with the partner support authority's role. Without it a partner credential runs as the partner organization, which outside the partner endpoints is deprecated (responses carry Deprecation and Sunset: Tue, 05 Jan 2027 00:00:00 GMT). A malformed value returns 400 INVALID_AUTH_ORG; a client organization the partner cannot access returns 403 PARTNER_CLIENT_ACCESS_DENIED.

Length: 22 to 22

Body

application/json · object

Request object for searching similar object embeddings by text.

includeVectorboolean | nullbodydefault: false

Whether to include the embedding vector in the response.

maxNumResultsinteger<int32> | nullbody

Optional number of results to return. Setting to larger numbers will increase latency.

modelenum<string>bodyrequired

Model used for embeddings. The streaming variant of this search accepts only "CHARON".

Available options: CLIP_512, CHARON

objectTypeFilterenum<string>[] | nullbody

Optional object type filter. Provide explicit type values to narrow results, or omit to search all object types. Pass the single value "AUTO" to have the server infer the object type (humans or vehicles) from the search text.

onlyMatchesboolean | nullbody

Only applies to models that classify results as matches: when true (default) return only actual matches; when false return all results within the similarity threshold.

queryDeviceUuidsstring<DeviceFacetUuid>[] | nullbody

Optional list of device uuids to filter result set on.

queryEndTimeMsinteger<int64> | nullbodyrequired

End time in epoch milliseconds. Results returned will have occurred before this time.

queryStartTimeMsinteger<int64> | nullbodyrequired

Start time in epoch milliseconds. Results returned will have occurred after this time.

regionFilterinteger<int32>[][] | nullbody

Optional region filter to return results only if they overlap with the supplied region. This is a list of perymyriad points defining a polygon. The first and last elements must be the same to make a connected polygon. For example, a triangle should have 4 points defined in this list.

searchTextstring | nullbodyrequired

The search text to find similar objects to. Phrases like "red sedan", "person in yellow shirt", etc.

similarityThresholdnumber<float> | nullbody

Optional threshold applied to filter returned results, set as a float in [0.0, 1.0].

bash

curl --request POST \
  --url 'https://api2.rhombussystems.com/api/search/searchSimilarObjectEmbeddingsByText' \
  --header 'x-auth-scheme: api-token' \
  --header 'x-auth-apikey: YOUR_API_KEY' \
  --header 'Content-Type: application/json' \
  --data '{
    "includeVector": false,
    "maxNumResults": 0,
    "model": "CLIP_512",
    "objectTypeFilter": [
      "MOTION_HUMAN"
    ],
    "onlyMatches": true,
    "queryDeviceUuids": [
      "AAAAAAAAAAAAAAAAAAAAAA.v0"
    ],
    "queryEndTimeMs": 0,
    "queryStartTimeMs": 0,
    "regionFilter": [
      [
        0
      ]
    ],
    "searchText": "<string>",
    "similarityThreshold": 0
  }'

import requests

url = "https://api2.rhombussystems.com/api/search/searchSimilarObjectEmbeddingsByText"
headers = {
    "x-auth-scheme": "api-token",
    "x-auth-apikey": "YOUR_API_KEY",
    "Content-Type": "application/json",
}
payload = {
    "includeVector": False,
    "maxNumResults": 0,
    "model": "CLIP_512",
    "objectTypeFilter": [
        "MOTION_HUMAN"
    ],
    "onlyMatches": True,
    "queryDeviceUuids": [
        "AAAAAAAAAAAAAAAAAAAAAA.v0"
    ],
    "queryEndTimeMs": 0,
    "queryStartTimeMs": 0,
    "regionFilter": [
        [
            0
        ]
    ],
    "searchText": "<string>",
    "similarityThreshold": 0
}

response = requests.post(url, headers=headers, json=payload)
response.raise_for_status()
print(response.json())

const response = await fetch('https://api2.rhombussystems.com/api/search/searchSimilarObjectEmbeddingsByText', {
  method: 'POST',
  headers: {
    'x-auth-scheme': 'api-token',
    'x-auth-apikey': 'YOUR_API_KEY',
    'Content-Type': 'application/json',
  },
  body: JSON.stringify({
    "includeVector": false,
    "maxNumResults": 0,
    "model": "CLIP_512",
    "objectTypeFilter": [
      "MOTION_HUMAN"
    ],
    "onlyMatches": true,
    "queryDeviceUuids": [
      "AAAAAAAAAAAAAAAAAAAAAA.v0"
    ],
    "queryEndTimeMs": 0,
    "queryStartTimeMs": 0,
    "regionFilter": [
      [
        0
      ]
    ],
    "searchText": "<string>",
    "similarityThreshold": 0
  }),
});
if (!response.ok) throw new Error(`${response.status}: ${await response.text()}`);

console.log(await response.json());

import java.net.URI;
import java.net.http.HttpClient;
import java.net.http.HttpRequest;
import java.net.http.HttpResponse;

public class Example {
    public static void main(String[] args) throws Exception {
        String body = """
            {
              "includeVector": false,
              "maxNumResults": 0,
              "model": "CLIP_512",
              "objectTypeFilter": [
                "MOTION_HUMAN"
              ],
              "onlyMatches": true,
              "queryDeviceUuids": [
                "AAAAAAAAAAAAAAAAAAAAAA.v0"
              ],
              "queryEndTimeMs": 0,
              "queryStartTimeMs": 0,
              "regionFilter": [
                [
                  0
                ]
              ],
              "searchText": "<string>",
              "similarityThreshold": 0
            }
            """;
        HttpRequest request = HttpRequest.newBuilder()
            .uri(URI.create("https://api2.rhombussystems.com/api/search/searchSimilarObjectEmbeddingsByText"))
            .header("x-auth-scheme", "api-token")
            .header("x-auth-apikey", "YOUR_API_KEY")
            .header("Content-Type", "application/json")
            .method("POST", HttpRequest.BodyPublishers.ofString(body))
            .build();
        HttpResponse<String> response = HttpClient.newHttpClient()
            .send(request, HttpResponse.BodyHandlers.ofString());
        System.out.println(response.statusCode() + " " + response.body());
    }
}

json

{
  "error": false,
  "errorCode": "<string>",
  "moderationReason": "<string>",
  "queryAllowed": true,
  "similarEmbeddings": [
    {
      "distance": 0,
      "embedding": {
        "b": 0,
        "clipEndMs": 0,
        "deviceUuid": "AAAAAAAAAAAAAAAAAAAAAA.v0",
        "embedding": [
          0
        ],
        "embeddingId": "<string>",
        "isMatch": "<string>",
        "l": 0,
        "locationUuid": "AAAAAAAAAAAAAAAAAAAAAA",
        "model": "CLIP_512",
        "objectId": 0,
        "objectType": "<string>",
        "orgUuid": "AAAAAAAAAAAAAAAAAAAAAA",
        "r": 0,
        "stableTrackId": 0,
        "t": 0,
        "thumbnailLocation": "<string>",
        "thumbnailUri": "<string>",
        "timestamp": 0
      },
      "matchedEmbeddingId": "AAAAAAAAAAAAAAAAAAAAAA"
    }
  ]
}

Response

200 OK

application/json · object

Response object for searching similar object embeddings.

errorboolean | null

errorCodestring | null

Stable, machine-readable code for the failure (one of the ApiErrorCode catalog) when error is true. Branch on this, not on errorMsg.

errorMsgstring | null

moderationReasonstring | null

Reason for moderation if query is not allowed

queryAllowedboolean | null

Whether the query is allowed

similarEmbeddingsobject[] | null

List of similar object embedding matches

Show child attributes

distancenumber<float> | null

embeddingobject

Show child attributes

binteger<int32> | null

bounding box - bottom permyriad

clipEndMsinteger<int64> | null

deviceUuidstring<DeviceFacetUuid> | null

RUUID with optional appended facet information

Example: "AAAAAAAAAAAAAAAAAAAAAA.v0"

embeddingnumber<float>[] | null

embeddingIdstring | null

isMatchstring | null

Match confidence for the query: "YES", "MAYBE", or "NO". Set on CHARON model results; null for CLIP.

linteger<int32> | null

bounding box - left permyriad

locationUuidstring<RUUID> | null

base 64 (url-safe) uuid string

Example: "AAAAAAAAAAAAAAAAAAAAAA"

modelenum<string>

Model used for embeddings. The streaming variant of this search accepts only "CHARON".

Available options: CLIP_512, CHARON

objectIdinteger<int32> | null

objectTypestring | null

orgUuidstring<RUUID> | null

base 64 (url-safe) uuid string

Example: "AAAAAAAAAAAAAAAAAAAAAA"

rinteger<int32> | null

bounding box - right permyriad

stableTrackIdinteger<int32> | null

tinteger<int32> | null

bounding box - top permyriad

thumbnailLocationstring | null

thumbnailUristring | null

timestampinteger<int64> | null

matchedEmbeddingIdstring<RUUID> | null

base 64 (url-safe) uuid string

Example: "AAAAAAAAAAAAAAAAAAAAAA"

warningMsgstring | null

Errors

Like every operation, this one can also answer with the standard error body (errorCode, errorMsg). Errors explains each status and how to retry.

400 Bad Request. errorCode is one of: BAD_REQUEST, VALIDATION_FAILED, MISSING_REQUIRED_FIELD, INVALID_JSON, INVALID_REQUEST_BODY, FEATURE_NOT_ENABLED, INVALID_AUTH_ORG, LOCATION_NOT_FOUND.
401 Unauthorized. errorCode is one of: UNAUTHENTICATED, INVALID_CREDENTIAL, CREDENTIAL_EXPIRED, CREDENTIAL_REVOKED, CERTIFICATE_MISMATCH.
403 Forbidden. errorCode is one of: FORBIDDEN, DOMAIN_NOT_ALLOWED, DEVICE_SCOPE_MISMATCH, ROLE_NOT_PERMITTED, FUNCTIONALITY_REQUIRED, PARTNER_FUNCTIONALITY_REQUIRED, PERMISSION_REQUIRED, PARTNER_CLIENT_ACCESS_DENIED, HARDWARE_NOT_AVAILABLE, HARDWARE_ALREADY_REGISTERED, PARTNER_PORTAL_NOT_ENABLED, MANAGED_BY_MSP, SUPER_ADMIN_RESTRICTED_BY_MSP, MSP_DESIGNATION_INVALID, NOT_MANAGED_CLIENT.
429 Too Many Requests. errorCode is one of: RATE_LIMITED.
500 Internal Server Error. errorCode is one of: INTERNAL_ERROR.
503 Service Unavailable. errorCode is one of: SERVICE_UNAVAILABLE.