NAV Navbar
java python typescript cURL

Dalet Media Cortex API

The Dalet Media Cortex API allows you to submit media analysis jobs managed by the Dalet Media Cortex service. Media analysis jobs include for example speech-to-text, face recognition, content moderation, OCR or brand detection.

The Dalet Cortex API is a REST API with typed schema for the payload.

In order to start using the Dalet Cortex service, you need to obtain a valid subscription. Start here to get your credentials for Dalet Media Cortex.

Architecture

Dalet Media Cortex job processing is performed on the cloud via dynamic combination of microservices. Dalet Media Cortex adopts the EBU MCMA architecture. The key objectives of this architecture are to support:

The Dalet Media Cortex architecture is implemented using the serverless approach - relying on independent microservices accessible through well documented REST endpoints and sharing a common object model.

Roles

The following services are involved in the processing of Dalet Media Cortex jobs exposed through the Dalet Media Cortex API:

  1. Checking authentication using an API key and a token mechanism
  2. Verifying quota restrictions before accepting a submitted job
  3. Keeping track of usage so that job processing can be tracked and billed
  4. Keeping track of jobs metadata as a job repository

When integrated with Dalet Galaxy, the Dalet Media Cortex API is invoked by a dedicated on-premise Media Cortex Indexer agent, which interacts with Dalet Media Cortex and orchestrates the exchange of media on cloud storage and the storage of the returned metadata in the Dalet Galaxy MAM repository.

Getting Started

Subscription

In order to start using the Dalet Mediator services, you need to obtain a valid subscription which provides personal client authorization keys. Start here to get one for Dalet Media Cortex.

We provide client SDK libraries to ease the interaction with the Dalet Media Cortex API in different programming languages (Python, TypeScript, Java). For each language, we also provide a sample application that demonstrates how to perform a media indexing job end to end. In the sample apps below, the credential keys are accessed under the names:

The sample apps reads these keys from a configuration file in a JSON format, for example:

{
      "clientKey": "clientKey1234",
      "clientSecret": "clientSecret1234",
      "projectServiceId": "projectServiceId1234",
      "bucketName": "my-bucket",
      "localPath": ".\Samples",
      "inputFile": "sample.mp4",
      "outputFile": "sample-output.json"
}

Installation

We describe below how to install the Dalet Media Cortex client library for each supported programming language:

Java

If you are using Maven, add the following to your pom.xml file:
<dependency>
      <groupId>com.dalet.mediator</groupId>
      <artifactId>cortex-client</artifactId>
      <version>1.2.0</version>
</dependency>

If you are using Gradle, add the following to your dependencies:
compile 'com.dalet.mediator:cortex-client:1.2.0'

If you are using SBT, add the following to your dependencies:
libraryDependencies += "com.dalet.mediator" % "cortex-client" % "1.2.0"

Python

For more information, see Setting Up a Python Development Environment.

Using pip:
pip install --upgrade git+https://github.com/daletcoreil/cortex-client-python-sdk.git

Using setuptools:

TypeScript

Using NPM repository:
npm install --save git+https://github.com/daletcoreil/cortex-client-tsnode-sdk.git

Using downloaded SDK:

Usage

Once you have a subscription and your platform is setup with a Dalet Media Cortex client SDK, you can start using Dalet Media Cortex services.

A typical Dalet Media Cortex session to submit and track a job follows these steps:

See Sample Application for more details.

Sample Application

The following complete API examples are provided in Java, Python and TypeScript and demonstrate end-to-end interactions with the Dalet Media Cortex API, from job preparation, job submission, job monitoring, up to displaying the received results.

Download the full sample files here:   Java  Python  TypeScript

Upload a media

Upload a media file to your AWS S3 bucket

TransferManager tm = TransferManagerBuilder
    .standard()
    .withS3Client(s3Client)
    .build();
Upload upload = tm.upload(bucketName, bucketKey, mediaFile);
upload.waitForCompletion();

s3 = boto3.client('s3', aws_access_key_id=aws_access_key_id, aws_secret_access_key=aws_secret_access_key, region_name='us-east-1', config=Config(s3={'addressing_style': 'path'}, signature_version='s3v4'))
s3.upload_file(inputFile, bucketName, bucketKey)
const credentials = new AWS.Credentials(aws_access_key_id, aws_secret_access_key);
const s3 = new AWS.S3({
    apiVersion: '2006-03-01',
    signatureVersion: 'v4',
    s3ForcePathStyle: true,
    credentials: credentials
});
const read: ReadStream = fs.createReadStream(inputMediaFilePath);
const params: AWS.S3.PutObjectRequest = {
        Bucket: bucketName,
        Key: ibucketInKey,
        Body: read
    };
await s3.upload(params).promise();
aws s3 cp FILE_PATH s3://BUCKET_NAME/

Acquire a signed URL

Acquire a signed URL which provides access to the media file located on the S3 bucket

 GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(bucket, key)
        .withMethod(HttpMethod.GET)
        .withExpiration(expiration));
 URL url = s3Client.generatePresignedUrl(request);
s3 = boto3.client('s3')
s3.generate_presigned_url(ClientMethod='get_object', Params={'Bucket': bucketName,'Key': bucketKey})
const credentials = new AWS.Credentials(aws_access_key_id, aws_secret_access_key);
const s3 = new AWS.S3({
    apiVersion: '2006-03-01',
    signatureVersion: 'v4',
    s3ForcePathStyle: true,
    credentials: credentials
});
const url: string = s3.getSignedUrl('getObject', {
        "Bucket": BucketName,
        "Key": BucketKey
    });
aws s3 presign s3://BUCKET_NAME/BUCKET_KEY

Prepare job

Prepare a Dalet Media Cortex job requesting the annotation of the media file using the Speech to Text service in the backend

Locator jsonFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_json)
        .httpEndpoint(outputSignedUrl_json);

Locator ttmlFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_ttml)
        .httpEndpoint(outputSignedUrl_ttml);

Locator textFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_text)
        .httpEndpoint(outputSignedUrl_text);

SpeechToTextOutput outputLocation = new SpeechToTextOutput().jsonFormat(jsonFormat).ttmlFormat(ttmlFormat).textFormat(textFormat);

JobInput spJobInput = new SpeechToTextInput()
        .inputFile(inputFile)
        .outputLocation(outputLocation)
        .language("Spanish")
        .title("1234");

Job aiJob = new Job()
        .jobType(Job.JobTypeEnum.AIJOB)
        .jobProfile(Job.JobProfileEnum.MEDIACORTEXSPEECHTOTEXT)
        .jobInput(spJobInput);

JobMediatorInput jobMediatorInput = new JobMediatorInput()
        .projectServiceId(project_service_id)
        .quantity(file_duration)
        .tracking("tracking")
        .job(aiJob);

input_file: Locator = Locator(bucketName, inputKey, signed_url_input)
jsonFormat: Locator = Locator(bucketName, outputKey_json, signed_url_otput_json)
ttmlFormat: Locator = Locator(bucketName, outputKey_ttml, signed_url_otput_ttml)
textFormat: Locator = Locator(bucketName, outputKey_text, signed_url_otput_text)
output_location: SpeechToTextOutput = SpeechToTextOutput(jsonFormat, ttmlFormat, None, textFormat)
job_input: SpeechToTextInput = SpeechToTextInput(input_file, output_location)
job: Job = Job(job_type='AIJob', job_profile='MediaCortexSpeechToText', job_input=job_input)
jobMediatorInput: JobMediatorInput = JobMediatorInput(project_service_id=project_service_id, quantity=30, job=job)
let inputFile: Locator  = new Locator();
inputFile.awsS3Bucket   = s3Bucket;
inputFile.awsS3Key      = inputMediaFile.name;
inputFile.httpEndpoint  = s3InputSignedUrl;

let jsonFormat: Locator  = new Locator();
jsonFormat.awsS3Bucket   = s3Bucket;
jsonFormat.awsS3Key      = outputFileName_json;
jsonFormat.httpEndpoint  = s3OutputSignedUrl_json;

let ttmlFormat: Locator  = new Locator();
ttmlFormat.awsS3Bucket   = s3Bucket;
ttmlFormat.awsS3Key      = outputFileName_ttml;
ttmlFormat.httpEndpoint  = s3OutputSignedUrl_ttml;

let textFormat: Locator  = new Locator();
textFormat.awsS3Bucket   = s3Bucket;
textFormat.awsS3Key      = outputFileName_text;
textFormat.httpEndpoint  = s3OutputSignedUrl_text;

let outputLocation: SpeechToTextOutput  = new SpeechToTextOutput();
outputLocation.jsonFormat = jsonFormat;
outputLocation.ttmlFormat = ttmlFormat;
outputLocation.textFormat = textFormat;

let jobInput: SpeechToTextInput = new SpeechToTextInput();
jobInput.jobInputType   = SpeechToTextInput.name;
jobInput.inputFile      = inputFile;
jobInput.outputLocation = outputLocation;

let job: Job  = new Job();
job.jobType     = Job.JobTypeEnum.AIJob;
job.jobProfile  = Job.JobProfileEnum.MediaCortexSpeechToText;
job.jobInput    = jobInput;

let jobMediatorInput: JobMediatorInput  = new JobMediatorInput();
jobMediatorInput.projectServiceId   = projectServiceId;
jobMediatorInput.quantity           = inputMediaFile.duration;
jobMediatorInput.job                = job;

Acquire an access token

Given your secret keys, acquire an access token to interact with the Dalet Media Cortex API

AuthApi apiInstance = new AuthApi(Configuration.getDefaultApiClient());
Token token = apiInstance.getAccessToken(client_id, client_secret);
api_instance: AuthApi = mediator_client.AuthApi(mediator_client.ApiClient(Configuration()))
api_instance.get_access_token(client, secret)
const authApi: AuthApi = new AuthApi();
await authApi.getAccessToken(client, secret)).body
curl -X POST \
  https://CORTEX_ADDRESS/auth/access-token \
  -H 'client: CORTEX_CLIENT_KEY' \
  -H 'secret: CORTEX_CLIENT_SECRET'

Submit job

Submit the job to the Dalet Media Cortex API

ApiClient apiClient = Configuration.getDefaultApiClient();
ApiKeyAuth tokenSignature = (ApiKeyAuth) apiClient.getAuthentication("tokenSignature");
tokenSignature.setApiKey(token.getAuthorization());

JobsApi jobsApi = new JobsApi(apiClient);
jobsApi.createJob(jobMediatorInput);
api_instance: JobsApi = mediator_client.JobsApi(mediator_client.ApiClient(Configuration()))
api_instance.create_job(job_mediator_input)
const submittedJob: MediatorJob = await jobsApi.createJob(jobInput)).body
curl -X POST \
  https://CORTEX_ADDRESS/jobs \
  -H 'Authorization: BEARER_TOKEN' \
  -H 'Content-Type: application/json' \
  -d '{
  "projectServiceId":"PROJECT_SERVICE_ID",
  "quantity": 30,
  "tracking": "EXTRA_CLIENT_INFORMATION",
  "job":   {
    "jobType": "AIJob",
    "jobProfile": "MediaCortexSpeechToText",
    "jobInput": {
      "jobInputType": "SpeechToTextInput",
      "inputFile": {
        "awsS3Bucket": "INPUT_BUCKET_NAME",
        "awsS3Key": "INPUT_BUCKET_KEY",
        "httpEndpoint": "INPUT_SIGNED_URL"
      },
      "outputLocation": {
         "jsonFormat": {
            "awsS3Bucket": "OUTPUT_BUCKET_NAME",
            "awsS3Key": "OUTPUT_BUCKET_KEY",
            "httpEndpoint": "OUTPUT_SIGNED_URL"
         }
      }
    }
  }
}
'

Monitor job status

Monitor the status of the job while it is progressing.
We recommend to perform this call at a frequency of no more than once every 30 seconds.

MediatorJob mediatorJob = jobsApi.getJobById(jobId);
JobMediatorStatus jobStatus = mediatorJob.getStatus();
api_instance: JobsApi = mediator_client.JobsApi(mediator_client.ApiClient(Configuration()))
mediator_job: MediatorJob = api_instance.get_job_by_id(job_id)
mediator_status: JobMediatorStatus = mediator_job.status
const job: MediatorJob = await jobsApi.getJobById(job.id)).body;
let mediatorStatus: JobMediatorStatus = job.status;
curl -X GET \
  https://CORTEX_ADDRESS/jobs/JOB_ID \
  -H 'Authorization: BEARER_TOKEN' \
  -H 'Content-Type: application/json'

Download results

Download the results of the media annotation after the job has completed

File localFile_json = new File(outputFile_json);
s3Client.getObject(new GetObjectRequest(bucketName, outputKey_json), localFile_json);

File localFile_ttml = new File(outputFile_ttml);
s3Client.getObject(new GetObjectRequest(bucketName, outputKey_ttml), localFile_ttml);

File localFile_text = new File(outputFile_text);
s3Client.getObject(new GetObjectRequest(bucketName, outputKey_text), localFile_text);       
s3 = boto3.client('s3')
s3.download_file(bucketName, outputKey_json, outputFile_json)
s3.download_file(bucketName, outputKey_ttml, outputFile_ttml)
s3.download_file(bucketName, outputKey_text, outputFile_text)
const params: AWS.S3.GetObjectRequest = {
    Bucket: bucketName,
    Key: bucketKey
};
const result = await s3.getObject(params).promise();
fs.writeFileSync(inputMediaFile.folder + outputFileName, result.Body);
aws s3 cp s3://OUTPUT_BUCKET_NAME/OUTPUT_BUCKET_KEY FILE_PATH

Delete results

Delete the JSON result file and the media on the S3 bucket after you have safely downloaded them

s3Client.deleteObject(bucketName, bucketKey);
s3 = boto3.client('s3')
s3.delete_object(Bucket=bucketName, Key=inputKey)
const inputParams: AWS.S3.DeleteObjectRequest = {
    Bucket: bucketName,
    Key: inBucketKey
};
const outputParams: AWS.S3.DeleteObjectRequest = {
    Bucket: bucketName,
    Key: outBucketKey
};
await Promise.all([
    s3.deleteObject(inputParams).promise(),
    s3.deleteObject(outputParams).promise()
]);
aws s3 rm s3://INPUT_BUCKET_NAME/INPUT_BUCKET_KEY
aws s3 rm s3://OUTPUT_BUCKET_NAME/OUTPUT_BUCKET_KEY

Authentication

To use the Dalet Media Cortex API - you must obtain an APIKey from Dalet. This key comes in the form of two parameters:

Given these two parameters, a client program must first obtain an access token (POST /auth/access-token) and then associate this token to all subsequent calls in a header. Dalet Cortex Media API expects the API key to be included in all API requests to the server in a header:

Authorization: meowmeowmeow

When the token expires, the API will return a 401 error code. In this case, the client must request a new token and resubmit the request.

Get authentication token

To get authentication token, use this code:

import com.dalet.mediator.cortex.ApiClient;
import com.dalet.mediator.cortex.ApiException;
import com.dalet.mediator.cortex.Configuration;
import com.dalet.mediator.cortex.api.AuthApi;
import com.dalet.mediator.cortex.model.Token;

ApiClient defaultClient = Configuration.getDefaultApiClient();
AuthApi apiInstance = new AuthApi(defaultClient);
Token token = apiInstance.getAccessToken('client_id', 'client_secret');
import cortex_client
from cortex_client.rest import ApiException

def main():
    try:
        api_instance = cortex_client.AuthApi(cortex_client.ApiClient())
        token = api_instance.get_access_token('client_id', 'client_secret')     
    except ApiException as e:
        print("Exception when calling api: %s\n" % e)
import {
    AuthApi,
    Token
} from 'mediator-cortex_client'

const client_id: string = 'myClientId';
const client_secret: string = 'mySecret';

const getAccessToken = async (): Promise<Token> => {
    return (await authApi.getAccessToken(client_id, client_secret)).body;
};

Make sure to replace client_id and client_secret with your client and secret values.

This endpoint retrieves the authentication token.

HTTP Request

POST /auth/access-token

Header Parameters

Parameter Description
client Authentication client
secret Authentication secret

Response

Response Description Body
200 Success response Token

Example:
{
      "Authorization": "Bearer eyJraWQiOiIwZ2FsdS1SQktGUFpPYlRmWFI0b05YVjZaVEt4dFEw",
      "Expires": 86400
}

Errors

Error Description
401 Invalid secret or client

Jobs

Submit a new job

To submit a new Dalet Media Cortex job, use this code:

import com.dalet.mediator.cortex.ApiClient;
import com.dalet.mediator.cortex.Configuration;
import com.dalet.mediator.cortex.model.*;

String bucket       = "my-bucket-name";
String inKey        = "00000C7B.mp4";
String inSignedUrl  = "https://my-bucket-name.s3.us-east-1.amazonaws.com/media/00000C7B.mp4";
String outputKey_json       = "00000C7B.json";
String outputKey_ttml       = "00000C7B.ttml";
String outputKey_text       = "00000C7B.txt";
String outSignedUrl_json = "https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.json";
String outSignedUrl_ttml = "https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.ttml";
String outSignedUrl_text = "https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.txt";

Locator jsonFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_json)
        .httpEndpoint(outputSignedUrl_json);

Locator ttmlFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_ttml)
        .httpEndpoint(outputSignedUrl_ttml);

Locator textFormat = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey_text)
        .httpEndpoint(outputSignedUrl_text);

SpeechToTextOutput outputLocation = new SpeechToTextOutput().jsonFormat(jsonFormat).ttmlFormat(ttmlFormat).textFormat(textFormat);      

JobInput spJobInput = new SpeechToTextInput()
        .inputFile(inputFile)
        .outputLocation(outputLocation)
        .language("Spanish")
        .title("1234");

Job aiJob = new Job()
        .jobType(Job.JobTypeEnum.AIJOB)
        .jobProfile(Job.JobProfileEnum.MEDIACORTEXSPEECHTOTEXT)
        .jobInput(spJobInput);

JobMediatorInput jobMediatorInput = new JobMediatorInput()
        .projectServiceId(project_service_id)
        .quantity(file_duration)
        .tracking("tracking")
        .job(aiJob);

ApiClient defaultClient = Configuration.getDefaultApiClient();
ApiKeyAuth tokenSignature = (ApiKeyAuth) defaultClient.getAuthentication("tokenSignature");
tokenSignature.setApiKey("myToken");
JobsApi apiInstance = new JobsApi(defaultClient);
MediatorJob job = apiInstance.createJob(jobMediatorInput);          
import time
import cortex_client
from cortex_client import Configuration, Locator, VideoAnalysisInput, AIJob, JobMediatorInput
from cortex_client.rest import ApiException

bucketName = 'com-dalet-cortex'
inputKey = 'upload_sample.mp4'
inputURL = 'https://com-my-bucket.s3.us-east-1.amazonaws.com/media/00000C7B.mp4'
outputKey_json      = '00000C7B.json'
outputKey_ttml      = '00000C7B.ttml'
outputKey_text      = '00000C7B.txt'
outSignedUrl_json = 'https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.json'
outSignedUrl_ttml = 'https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.ttml'
outSignedUrl_text = 'https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.txt'

def main():
    try:
        input_file: Locator = Locator(bucketName, inputKey, signed_url_input)
        jsonFormat: Locator = Locator(bucketName, outputKey_json, signed_url_otput_json)
        ttmlFormat: Locator = Locator(bucketName, outputKey_ttml, signed_url_otput_ttml)
        textFormat: Locator = Locator(bucketName, outputKey_text, signed_url_otput_text)

        output_location: SpeechToTextOutput = SpeechToTextOutput(jsonFormat, ttmlFormat, None, textFormat)
        job_input: SpeechToTextInput = SpeechToTextInput(input_file, output_location)

        job = Job(job_type='AIJob', job_profile='MediaCortexSpeechToText', job_input=job_input)
        job_mediator_input = JobMediatorInput(project_service_id='my project service id', \
        tracking='tracking', quantity=30, job=job)

        api_instance = cortex_client.JobsApi(cortex_client.ApiClient(mediator_config))
        mediator_job = api_instance.create_job(job_mediator_input)  
    except ApiException as e:
        print("Exception when calling api: %s\n" % e)       
import {
    JobMediatorInput,
    Locator,
    SpeechToTextInput,
    AIJob,
    MediatorJob,
    JobsApi,
    JobMediatorStatus,
    JobsApiApiKeys
} from 'cortex-client'

const submitJob = async (jobInput: JobMediatorInput): Promise<MediatorJob> => {
    return (await jobsApi.createJob(jobInput)).body;
};

Make sure to replace input and output parameters by your values.

This endpoint submits a job to Dalet Media Cortex.

HTTP Request

POST /jobs

Header Parameters

Parameter Description
Authorization Authentication token

Request body

JSON object of type JobMediatorInput describing the request to be executed.

Response

Response Description Body
201 Success response MediatorJob

Errors

Error Description
400 Bad request
401 Authorization request fail
502 Failed request. Reason is most likely quota violation.

Get job status

To get a job status, use this code:

import com.dalet.mediator.cortex.ApiClient;
import com.dalet.mediator.cortex.ApiException;
import com.dalet.mediator.cortex.Configuration;
import com.dalet.mediator.cortex.model.*;

ApiClient defaultClient = Configuration.getDefaultApiClient();
ApiKeyAuth tokenSignature = (ApiKeyAuth) defaultClient.getAuthentication("tokenSignature");
tokenSignature.setApiKey("myToken");

JobsApi apiInstance = new JobsApi(defaultClient);
MediatorJob mediatorJob = apiInstance.getJobById(job_id);
import time
import cortex_client
from cortex_client import Configuration, Locator, VideoAnalysisInput, AIJob, JobMediatorInput
from cortex_client.rest import ApiException

mediator_config = Configuration()
# set real authentication endpoint path
mediator_config.host = 'https://u9itw7ttvj.execute-api.us-east-1.amazonaws.com/test'

def wait_for_complete(job_id):
    mediator_job = get_mediator_job(job_id)
    status = mediator_job.status.status
    while status not in ["COMPLETED", "FAILED"]:
        time.sleep(30)
        mediator_job = get_mediator_job(mediator_job.id)
        status = mediator_job.status.status
    return mediator_job

def get_mediator_job(job_id):
    # create an instance of the API class
    api_instance = cortex_client.JobsApi(cortex_client.ApiClient(mediator_config))
    return api_instance.get_job_by_id(job_id)
import {
    MediatorJob,
    JobsApi,
    JobMediatorStatus,
    JobsApiApiKeys
} from 'cortex-client'

job = (await jobsApi.getJobById(job_id)).body;

Make sure to replace job_idwith your posted job id.

This endpoint gets the status of a running job.

HTTP Request

GET /jobs/{jobId}

Path Parameters

Parameter Description
jobId ID of the job as returned in JobMediatorEntity

Header Parameters

Parameter Description
Authorization Authentication token

Response

Response Description Body
200 Success response MediatorJob

Errors

Error Description
404 Job with required ID does not exist



Models

The Dalet Media Cortex API uses the following models.

For all objects defined in these models, mandatory fields are marked in bold.

Token

Name Type Description
Authorization string Autorization token
Expires integer Expires interval in seconds
default: 86400

An access token is necessary in order to submit requests to the API.

JobMediatorStatus

Name Type Description
status string Job status. Can be one of: "NEW", "FAILED", "QUEUED", "SCHEDULED", "RUNNING", "COMPLETED"
statusMessage string Last message reported by Job Processor handling this job.
progress integer Job progress in percent (from 0 to 100)

Current status of the job as reported by the associated Job Processor if the job is committed. Returned by the GET /jobs/{jobId} call.

JobOutput

Name Type Description
jobOutputType string Type of AI job

Where the output of the job analysis will be stored. Can be either VideoAnalysisOutput or SpeechToTextOutput.

Locator

Name Type Description
awsS3Bucket string Amazon S3 bucket name
awsS3Key string Media file name in Amazon S3 bucket
httpEndpoint string Signed URL to the media file in Amazon S3 bucket

Generic description of a file location according to the EBU FIMS specification.

SpeechToTextOutput

Name Type Description
jsonFormat Locator Location of the output file into which metadata will be stored in JSON format. The JSON format provides a timestamp for each transcription
ttmlFormat Locator Location of the output file into which the speech transcription will be stored as subtitles encoded in EBU-TT (TTML) format. See here
srtFormat Locator Location of the output file into which the speech transcription will be stored as subtitles encoded in SRT format. See here
textFormat Locator Location of the output file into which the text of the speech transcription will be stored in simple format with no timestamps

Specifies where the captions computed by the Speech-to-text service are to be stored. Different formats can be requested (EBU-TT, JSON or text).

MediatorJob

Name Type Description
id string Job ID computed by the Dalet Media Mediator Job registry. Uniquely identifies the job after it has been accepted by the mediator.
dateCreated string Datetime when the job was accepted by the Mediator
dateModified string Datetime when the job was last updated by a Job Processor service reporting progress on this job.
quantity integer Verified number of units after the job is completed. Units depend on the job profile (usually, they correspond to the duration of the media). Invoicing is based on this value, which may be different from the one submitted by the client.
status JobMediatorStatus Current status of the job as reported by the associated Job Processor if the job is committed.
jobOutput JobOutput Where the output of the job analysis will be stored. Can be either VideoAnalysisOutput or SpeechToTextOutput.

After a job is submitted to the Dalet Media Mediator service, it is attributed metadata that helps track its lifecycle. In addition to the input field (Job input / Job output), the JobMediatorEntity provides a unique ID, date created and last modified and usage information (verified quantity and usage ID to reconcile with invoice). These additional fields are all computed by the mediator and read-only.

JobInput

Name Type Description
jobInputType string Job input type

Can be either VideoAnalysisInput or SpeechToTextInput

Job

Name Type Description
jobType string Job type (should be AIJob)
jobProfile string Define what type of job is to be executed on the media object.
jobInput JobInput Can be either VideoAnalysisInput or SpeechToTextInput

Specific job description according to the job profile.

JobMediatorInput

Name Type Description
job Job Specific job description according to the job profile.
projectServiceId string Customer Project ID. This ID must be provided by Dalet when a Cortex service is provisioned. It must match the authorization token you have been provided.
quantity integer Number of units that will be charged for this job. The unit depends on the job profile. For example, for an AI metadata extraction job, units are seconds of media duration.
tracking JSON Client metadata associated to the job. This can include any identification fields provided by the client. It should identify the job in a unique manner and is useful to reconcile usage reports with client metadata.
notificationEndpoint string Callback URL endpoint to be called once the stage of the job is changed.

Toplevel description of any job submitted to the Dalet Media Mediator service. Encapsulates a service specific job description with authentication, billing and tracking metadata.

VideoAnalysisInput

Name Type Description
jobInputType string Job input type (should be VideoAnalysisInput)
inputFile Locator Input media file location info
outputLocation Locator Output result file location info
dataFilter string[] Determine which types of AI analyses are to be indexed. Possible values: audioEffects, brands, faces, keywords, labels, ocr,
sentiments, shots, textualContentModeration, transcript, visualContentModeration
language string Language of the media file to be indexed according to ISO 639-1 Language Name.
See here
startOfMaterial number Indicate the timecode of the first frame in the media in seconds. Generated subtitles start from this time. Example: 182.6

Describe the input and expected output of the job to be executed by Video Analise service. Files are specified using FIMS Essence Locators.

SpeechToTextInput

Name Type Description
jobInputType string Job input type (should be SpeechToTextInput)
inputFile Locator Input media file location info
outputLocation SpeechToTextOutput Output result file location info
language string Language of the media file to be indexed according to ISO 639-1 Language Name.
See here
title string Readable name associated to the job when submitted to Speech-to-text service
startOfMaterial number Indicate the timecode of the first frame in the media in seconds. Generated subtitles start from this time. Example: 182.6
vocabulary SpeechToTextVocabulary Custom dictionary

Describe the input and expected output of the job to be executed by Speech-to-text service. Files are specified using FIMS Essence Locators: which media file is to be indexed by the Speech-to-text service, its language, and a user defined name associated to the job. This job produces captions for the media in EBU-TT, JSON and textual formats

SpeechToTextVocabulary

Name Type Description
content string content
sounds_like string[] list of custom words or phrases that should be recognized

This is a list of custom words or phrases that should be recognized. Custom Dictionary Sounds is an extension to this to allow alternative pronunciations to be specified in order to aid recognition, or provide for alternative transcriptions

Legal Notice

Dalet Media Cortex API documentation
Revision: 1.0
Creation: 2019-05-31
This API document explains the basic use of the Dalet Media Cortex API for the management of automatic media indexing jobs.
(c) 2019, Dalet, all rights reserved

THIS API SPECIFICATION IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE API OR THE USE OR OTHER DEALINGS IN THE API.