NAV Navbar
java python typescript

Scope

Dalet Mediator API allows you to submit long running media jobs managed by Dalet services. Long running media jobs include:

The Dalet Mediator API is a REST API with typed schema for the payload.

In order to start using the Dalet Mediator services, you need to obtain a valid subscription. Start here to get one for Dalet Media Cortex.

Architecture

Job processing is performed on the cloud via dynamic combination of microservices. Dalet Mediator adopts the EBU MCMA architecture. The key objectives of this architecture are to support:

The architecture is implemented using the serverless approach - relying on independent microservices accessible through well documented REST endpoints and sharing a common object model.

Roles

The following services are involved in the processing of media jobs exposed through the Dalet Media Mediator API:

  1. Checking authentication using an API key and a token mechanism
  2. Verifying quota restrictions before accepting a submitted job
  3. Keeping track of usage so that job processing can be tracked and billed
  4. Keeping track of jobs metadata as a job repository

Getting Started

Subscription

In order to start using the Dalet Mediator services, you need to obtain a valid subscription which provides personal client authorization keys. Start here to get one for Dalet Media Cortex.

In the sample apps below, these keys are used under the names:

Installation

Installing the Cortex client library

Java

If you are using Maven, add the following to your pom.xml file:
<dependency>
      <groupId>com.dalet.mediator</groupId>
      <artifactId>cortex-client</artifactId>
      <version>1.2.0</version>
</dependency>

If you are using Gradle, add the following to your dependencies:
compile 'com.dalet.mediator:cortex-client:1.2.0'

If you are using SBT, add the following to your dependencies:
libraryDependencies += "com.dalet.mediator" % "cortex-client" % "1.0.3"

Python

For more information, see Setting Up a Python Development Environment.

Using pip:
pip install --upgrade git+https://github.com/daletcoreil/cortex-client-python-sdk.git

Using setuptools:

TypeScript

Using NPM repository:
npm install --save git+https://github.com/daletcoreil/cortex-client-tsnode-sdk.git

Using downloaded SDK:

Usage

Once you have a subscription and your platform is setup with Cortex client SDK, you can start using Cortex services.

A typical Mediator session to submit and track a job would look like this:

See Sample Application section for more usage information.

Sample Application

Included are complete API examples in Java, Python and TypeScript that demonstrate end-to-end interactions with the Dalet Mediator API, from job preparation, job submission, job monitoring, and to displaying the received results.

Download the full sample files here:   Java  Python  TypeScript

Upload a media

Upload a media file to your AWS S3 bucket

TransferManager tm = TransferManagerBuilder
    .standard()
    .withS3Client(s3Client)
    .build();
Upload upload = tm.upload(bucketName, bucketKey, mediaFile);
upload.waitForCompletion();

s3 = boto3.client('s3')
s3.upload_file(inputFile, bucketName, bucketKey)
const s3 = new AWS.S3({
    apiVersion: '2006-03-01',
    s3ForcePathStyle: true
});
const read: ReadStream = fs.createReadStream(inputMediaFilePath);
const params: AWS.S3.PutObjectRequest = {
        Bucket: bucketName,
        Key: ibucketInKey,
        Body: read
    };
await s3.upload(params).promise();

Acquire a signed URL

Acquire a signed URL which provides access to the media file located on the S3 bucket

 GeneratePresignedUrlRequest request = new GeneratePresignedUrlRequest(bucket, key)
        .withMethod(HttpMethod.GET)
        .withExpiration(expiration));
 URL url = s3Client.generatePresignedUrl(request);
s3 = boto3.client('s3')
s3.generate_presigned_url(ClientMethod='get_object', Params={'Bucket': bucketName,'Key': bucketKey})
const s3 = new AWS.S3({
    apiVersion: '2006-03-01',
    s3ForcePathStyle: true
});
const url: string = s3.getSignedUrl('getObject', {
        "Bucket": BucketName,
        "Key": BucketKey
    });

Prepare job

Prepare a Dalet Media Cortex job requesting the annotation of the media file using Video Analysis service in the backend

Locator inputFile = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(inputKey)
        .httpEndpoint(inputSignedUrl);

Locator outputLocation = new Locator()
        .awsS3Bucket(bucketName)
        .awsS3Key(outputKey)
        .httpEndpoint(outputSignedUrl);

JobInput viJobInput = new VideoAnalysisInput()
        .inputFile(inputFile)
        .outputLocation(outputLocation)
        .language(language)
        .dataFilter(Arrays.asList(VideoAnalysisInput.DataFilterEnum.FACES));

AIJob aiJob = new AIJob()
        .jobProfile(AIJob.JobProfileEnum.MEDIACORTEXVIDEOANALYSIS)
        .jobInput(viJobInput);

JobMediatorInput jobMediatorInput = new JobMediatorInput()
        .projectServiceId(project_service_id)
        .quantity(file_duration)
        .job(aiJob);
input_file: Locator = Locator(bucketName, bucketInKey, signed_url_input)
output_location: Locator = Locator(bucketName, bucketOutKey, signed_url_otput)
job_input: VideoAnalysisInput = VideoAnalysisInput(input_file, output_location)
job: AIJob = AIJob(job_profile='MediaCortexVideoAnalysis', job_input=job_input)
jobMediatorInput: JobMediatorInput = JobMediatorInput(project_service_id=project_service_id, quantity=30, job=job)
let inputFile: Locator  = new Locator();
inputFile.awsS3Bucket   = bucketName;
inputFile.awsS3Key      = inputBucketKey;
inputFile.httpEndpoint  = inputSignedUrl;

let outputLocation: Locator  = new Locator();
outputLocation.awsS3Bucket   = bucketName;
outputLocation.awsS3Key      = outputBucketKey;
outputLocation.httpEndpoint  = outputSignedUrl;

let jobInput: VideoAnalysisInput = new VideoAnalysisInput();
jobInput.jobInputType   = VideoAnalysisInput.name;
jobInput.inputFile      = inputFile;
jobInput.outputLocation = outputLocation;

let job: AIJob  = new AIJob();
job.jobType     = AIJob.name;
job.jobProfile  = AIJob.JobProfileEnum.MediaCortexVideoAnalysis;
job.jobInput    = jobInput;

let jobMediatorInput: JobMediatorInput  = new JobMediatorInput();
jobMediatorInput.projectServiceId   = projectServiceId;
jobMediatorInput.quantity           = inputMediaFile.duration;
jobMediatorInput.job                = job;

Acquire an access token

Given your secret keys, acquire an access token to interact with the Dalet Mediator API

AuthApi apiInstance = new AuthApi(Configuration.getDefaultApiClient());
Token token = apiInstance.getAccessToken(client_id, client_secret);
api_instance: AuthApi = mediator_client.AuthApi(mediator_client.ApiClient(Configuration()))
api_instance.get_access_token(client, secret)
const authApi: AuthApi = new AuthApi();
await authApi.getAccessToken(client, secret)).body

Submit job

Submit the job to the Dalet Mediator API

ApiClient apiClient = Configuration.getDefaultApiClient();
ApiKeyAuth tokenSignature = (ApiKeyAuth) apiClient.getAuthentication("tokenSignature");
tokenSignature.setApiKey(token.getAuthorization());

JobsApi jobsApi = new JobsApi(apiClient);
jobsApi.createJob(jobMediatorInput);
api_instance: JobsApi = mediator_client.JobsApi(mediator_client.ApiClient(Configuration()))
api_instance.create_job(job_mediator_input)
const submittedJob: MediatorJob = await jobsApi.createJob(jobInput)).body

Monitor job status

Monitor the status of the job while it is progressing

MediatorJob mediatorJob = jobsApi.getJobById(jobId);
JobMediatorStatus jobStatus = mediatorJob.getStatus();
api_instance: JobsApi = mediator_client.JobsApi(mediator_client.ApiClient(Configuration()))
mediator_job: MediatorJob = api_instance.get_job_by_id(job_id)
mediator_status: JobMediatorStatus = mediator_job.status
const job: MediatorJob = await jobsApi.getJobById(job.id)).body;
let mediatorStatus: JobMediatorStatus = job.status;

Download results

Download the results of the media annotation after the job has completed

s3Client.getObject(new GetObjectRequest(bucketName, bucketKey), outputFile);
s3 = boto3.client('s3')
s3.download_file(bucketName, buketOutKey, outputFile)
const params: AWS.S3.GetObjectRequest = {
    Bucket: bucketName,
    Key: bucketKey
};
const result = await s3.getObject(params).promise();
fs.writeFileSync(inputMediaFile.folder + outputFileName, result.Body);

Delete results

Delete the JSON result file and the media on the S3 bucket

s3Client.deleteObject(bucketName, bucketKey);
s3 = boto3.client('s3')
s3.delete_object(Bucket=bucketName, Key=inputKey)
const inputParams: AWS.S3.DeleteObjectRequest = {
    Bucket: bucketName,
    Key: inBucketKey
};
const outputParams: AWS.S3.DeleteObjectRequest = {
    Bucket: bucketName,
    Key: outBucketKey
};
await Promise.all([
    s3.deleteObject(inputParams).promise(),
    s3.deleteObject(outputParams).promise()
]);

Authentication

To use the Dalet Cortex API - you must obtain an APIKey from Dalet. This key comes in the form of two parameters:

Given these two parameters, a client program must first obtain an access token (GET /auth/access-token) and then associate this token to all subsequent calls in a header. Dalet Mediator API expects the API key to be included in all API requests to the server in a header:

Authorization: meowmeowmeow

When the token expires, the API will return a 401 error code. In this case, the client must request a new token and resubmit the request.

Get authentication token

To get authentication token, use this code:

import com.dalet.mediator.cortex.ApiClient;
import com.dalet.mediator.cortex.ApiException;
import com.dalet.mediator.cortex.Configuration;
import com.dalet.mediator.cortex.api.AuthApi;
import com.dalet.mediator.cortex.model.Token;

ApiClient defaultClient = Configuration.getDefaultApiClient();
AuthApi apiInstance = new AuthApi(defaultClient);
Token token = apiInstance.getAccessToken('client_id', 'client_secret');
import cortex_client
from cortex_client.rest import ApiException

def main():
    try:
        api_instance = cortex_client.AuthApi(cortex_client.ApiClient())
        token = api_instance.get_access_token('client_id', 'client_secret')     
    except ApiException as e:
        print("Exception when calling api: %s\n" % e)
import {
    AuthApi,
    Token
} from 'mediator-cortex_client'

const client_id: string = 'myClientId';
const client_secret: string = 'mySecret';

const getAccessToken = async (): Promise<Token> => {
    return (await authApi.getAccessToken(client_id, client_secret)).body;
};

Make sure to replace client_id and client_secret with your client and secret values.

This endpoint retrieves the authentication token.

HTTP Request

GET /jobs/token

Header Parameters

Parameter Description
client Authentication client
secret Authentication secret

Response

Response Description Body
200 Success response Token

Example:
{
      "Authorization": "Bearer eyJraWQiOiIwZ2FsdS1SQktGUFpPYlRmWFI0b05YVjZaVEt4dFEw",
      "Expires": 86400
}

Errors

Error Description
401 Invalid secret or client

Jobs

Submit a new job

To submin a new mediator job , use this code:

import com.dalet.mediator.cortex.ApiClient;
import com.dalet.mediator.cortex.Configuration;
import com.dalet.mediator.cortex.model.*;

String bucket       = "my-bucket-name";
String inKey        = "00000C7B.mp4";
String inSignedUrl  = "https://my-bucket-name.s3.us-east-1.amazonaws.com/media/00000C7B.mp4";
String outKey       = "00000C7B.json";
String outSignedUrl = "https://my-bucket-name.s3.us-east-1.amazonaws.com/result/00000C7B.json";

Locator inputFile = new Locator()
        .awsS3Bucket(bucket)
        .awsS3Key(inKey)
        .httpEndpoint(inSignedUrl);
Locator outputLocation = new Locator()
        .awsS3Bucket(bucket)
        .awsS3Key(outKey)
        .httpEndpoint(outSignedUrl);
JobInput viJobInput = new VideoAnalysisInput()
        .inputFile(inputFile)
        .outputLocation(outputLocation)
        .language("english")
        .dataFilter(new ArrayList<VideoAnalysisInput.DataFilterEnum>() {{ 
            add(VideoAnalysisInput.DataFilterEnum.FACES);}}
        );      
AIJob aiJob = new AIJob()
        .jobProfile(AIJob.JobProfileEnum.MEDIACORTEXVIDEOANALISIS)
        .jobInput(viJobInput);
JobMediatorInput jobMediatorInput = new JobMediatorInput()
        .projectServiceId("my project service id")
        .quantity(30)
        .job(aiJob);

ApiClient defaultClient = Configuration.getDefaultApiClient();
ApiKeyAuth tokenSignature = (ApiKeyAuth) defaultClient.getAuthentication("tokenSignature");
tokenSignature.setApiKey("myToken");
JobsApi apiInstance = new JobsApi(defaultClient);
MediatorJob job = apiInstance.createJob(jobMediatorInput);          
import time
import cortex_client
from cortex_client import Configuration, Locator, VideoAnalysisInput, AIJob, JobMediatorInput
from cortex_client.rest import ApiException

bucketName = 'com-dalet-cortex'
inputKey = 'upload_sample.mp4'
inputURL = 'https://com-my-bucket.s3.us-east-1.amazonaws.com/media/00000C7B.mp4'
outputKey = 'upload_sample.json'
outputURL = 'https://com-my-bucket.s3.us-east-1.amazonaws.com/00000C7B.json'

def main():
    try:
        mediator_config = Configuration()
        input_file = Locator(bucketName, inputKey,inputURL)
        output_location = Locator(bucketName, outputKey, outputURL)
        job_input = VideoAnalysisInput(input_file, output_location)
        job = AIJob(job_profile='MediaCortexVideoAnalysis', job_input=job_input)
        job_mediator_input = JobMediatorInput(project_service_id='my project service id', \
        tracking='tracking', quantity=30, job=job)

        api_instance = cortex_client.JobsApi(cortex_client.ApiClient(mediator_config))
        mediator_job = api_instance.create_job(job_mediator_input)  
    except ApiException as e:
        print("Exception when calling api: %s\n" % e)       
import {
    JobMediatorInput,
    Locator,
    VideoAnalysisInput,
    AIJob,
    MediatorJob,
    JobsApi,
    JobMediatorStatus,
    JobsApiApiKeys
} from 'cortex-client'

const submitJob = async (jobInput: JobMediatorInput): Promise<MediatorJob> => {
    return (await jobsApi.createJob(jobInput)).body;
};

Make sure to replace input and output parameters by your values.

This endpoint submits a job to Dalet Media Mediator.

HTTP Request

POST /jobs

Header Parameters

Parameter Description
Authorization Authentication token

Request body

JSON object of type JobMediatorInput describing the request to be executed.

Response

Response Description Body
201 Success response MediatorJob

Errors

Error Description
400 Bad request
401 Authorization request fail
502 Failed request. Reason is most likely quota violation.

Get job status

To get job status, use this code:

import com.dalet.mediator.cortex.ApiClient;
import com.dalet.mediator.cortex.ApiException;
import com.dalet.mediator.cortex.Configuration;
import com.dalet.mediator.cortex.model.*;

ApiClient defaultClient = Configuration.getDefaultApiClient();
ApiKeyAuth tokenSignature = (ApiKeyAuth) defaultClient.getAuthentication("tokenSignature");
tokenSignature.setApiKey("myToken");

JobsApi apiInstance = new JobsApi(defaultClient);
MediatorJob mediatorJob = apiInstance.getJobById(job_id);
import time
import cortex_client
from cortex_client import Configuration, Locator, VideoAnalysisInput, AIJob, JobMediatorInput
from cortex_client.rest import ApiException

mediator_config = Configuration()
# set real authentication endpoint path
mediator_config.host = 'https://u9itw7ttvj.execute-api.us-east-1.amazonaws.com/test'

def wait_for_complete(job_id):
    mediator_job = get_mediator_job(job_id)
    status = mediator_job.status.status
    while status not in ["COMPLETED", "FAILED"]:
        time.sleep(30)
        mediator_job = get_mediator_job(mediator_job.id)
        status = mediator_job.status.status
    return mediator_job

def get_mediator_job(job_id):
    # create an instance of the API class
    api_instance = cortex_client.JobsApi(cortex_client.ApiClient(mediator_config))
    return api_instance.get_job_by_id(job_id)
import {
    MediatorJob,
    JobsApi,
    JobMediatorStatus,
    JobsApiApiKeys
} from 'cortex-client'

job = (await jobsApi.getJobById(job_id)).body;

Make sure to replace job_idwith your posted job id.

This endpoint gets the status of a running job.

HTTP Request

GET /jobs/{jobId}

Path Parameters

Parameter Description
jobId ID of the job as returned in JobMediatorEntity

Response

Response Description Body
200 Success response MediatorJob

Errors

Error Description
404 Job with required ID does not exist

Models

The Dalet Media Mediator API uses the following models:

Token

Name Type Description
Authorization string Autorization token
Expires integer Expires interval in seconds
default: 86400

An access token is necessary in order to submit requests to the API.

JobMediatorStatus

Name Type Description
status string Job status. Can be one of: "NEW", "FAILED", "QUEUED", "SCHEDULED", "RUNNING", "COMPLETED"
statusMessage string Last message reported by Job Processor handling this job.
progress integer Job progress in percent (from 0 to 100)

Current status of the job as reported by the associated Job Processor if the job is committed. Returned by the GET /jobs/{jobId} call.

JobOutput

Name Type Description
jobOutputType string Type of AI job

Where the output of the job analysis will be stored. Can be either VideoAnalysisOutput or SpeechToTextOutput.

Locator

Name Type Description
awsS3Bucket string Amazon S3 bucket name
awsS3Key string Media file name in Amazon S3 bucket
httpEndpoint string Signed URL to the media file in Amazon S3 bucket

Generic description of a file location according to the EBU FIMS specification.

SpeechToTextOutput

Name Type Description
jsonFormat Locator Location of the output file into which metadata will be stored in JSON format. The JSON format provides a timestamp for each transcription
ttmlFormat Locator Location of the output file into which the speech transcription will be stored as subtitles encoded in EBU-TT (TTML) format. See here
srtFormat Locator Location of the output file into which the speech transcription will be stored as subtitles encoded in SRT format. See here
textFormat Locator Location of the output file into which the text of the speech transcription will be stored in simple format with no timestamps

Specifies where the captions computed by the Speech-to-text service are to be stored. Different formats can be requested (EBU-TT, JSON or text).

MediatorJob

Name Type Description
id string Job ID computed by the Dalet Media Mediator Job registry. Uniquely identifies the job after it has been accepted by the mediator.
dateCreated string Datetime when the job was accepted by the Mediator
dateModified string Datetime when the job was last updated by a Job Processor service reporting progress on this job.
quantity integer Verified number of units after the job is completed. Units depend on the job profile (usually, they correspond to the duration of the media). Invoicing is based on this value, which may be different from the one submitted by the client.
status JobMediatorStatus Current status of the job as reported by the associated Job Processor if the job is committed.
jobOutput JobOutput Where the output of the job analysis will be stored. Can be either VideoAnalysisOutput or SpeechToTextOutput.

After a job is submitted to the Dalet Media Mediator service, it is attributed metadata that helps track its lifecycle. In addition to the input field (Job input / Job output), the JobMediatorEntity provides a unique ID, date created and last modified and usage information (verified quantity and usage ID to reconcile with invoice). These additional fields are all computed by the mediator and read-only.

JobInput

Name Type Description
jobInputType string Job input type

Can be either VideoAnalysisInput or SpeechToTextInput

AIJob

Name Type Description
jobType string Job type (should be AIJob)
jobProfile string Define what type of job is to be executed on the media object.
jobInput JobInput Can be either VideoAnalysisInput or SpeechToTextInput

Specific job description according to the job profile.

JobMediatorInput

Name Type Description
job AIJob Specific job description according to the job profile.
projectServiceId string Customer Project ID. This ID must be provided by Dalet when a Cortex service is provisioned. It must match the authorization token you have been provided.
quantity integer Number of units that will be charged for this job. The unit depends on the job profile. For example, for an AI metadata extraction job, units are seconds of media duration.
tracking JSON Client metadata associated to the job. This can include any identification fields provided by the client. It should identify the job in a unique manner and is useful to reconcile usage reports with client metadata.
notificationEndpoint string Callback URL endpoint to be called once the stage of the job is changed.

Toplevel description of any job submitted to the Dalet Media Mediator service. Encapsulates a service specific job description with authentication, billing and tracking metadata.

VideoAnalysisInput

Name Type Description
jobInputType string Job input type (should be VideoAnalysisInput)
inputFile Locator Input media file location info
outputLocation Locator Output result file location info
dataFilter string[] Determine which types of AI analyses are to be indexed. Possible values: audioEffects, brands, faces, keywords, labels, ocr,
sentiments, shots, textualContentModeration, transcript, visualContentModeration
language string Language of the media file to be indexed according to ISO 639-1 Language Name.
See here
startOfMaterial number Indicate the timecode of the first frame in the media in seconds. Generated subtitles start from this time. Example: 182.6

Describe the input and expected output of the job to be executed by Video Analise service. Files are specified using FIMS Essence Locators.

SpeechToTextInput

Name Type Description
jobInputType string Job input type (should be SpeechToTextInput)
inputFile Locator Input media file location info
outputLocation SpeechToTextOutput Output result file location info
title string Readable name associated to the job when submitted to Speech-to-text service
startOfMaterial number Indicate the timecode of the first frame in the media in seconds. Generated subtitles start from this time. Example: 182.6
vocabulary SpeechToTextVocabulary Custom dictionary

Describe the input and expected output of the job to be executed by Speech-to-text service. Files are specified using FIMS Essence Locators: which media file is to be indexed by the Speech-to-text service, its language, and a user defined name associated to the job. This job produces captions for the media in EBU-TT, JSON and textual formats

SpeechToTextVocabulary

Name Type Description
content string content
sounds_like string[] list of custom words or phrases that should be recognized

This is a list of custom words or phrases that should be recognized. Custom Dictionary Sounds is an extension to this to allow alternative pronunciations to be specified in order to aid recognition, or provide for alternative transcriptions

Legal Notice

Dalet Mediator API documentation
Revision: 1.0
Creation: 2019-05-31
This API document explains the basic use of the Dalet Mediator API for long running job management.
(c) 2019, Dalet, all rights reserved

THIS API SPECIFICATION IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE API OR THE USE OR OTHER DEALINGS IN THE API.