Build with the Image API.

Discovery and accounting

Assets

Asynchronous jobs

Operation-specific job routes

Synchronous image routes

Local development transfer only

Brand colours: apply a 60–30–10 palette

POST /v1/images/brand-colors creates an asynchronous image.brand job from an uploaded asset. Supply one to four six-digit hex neutrals, one primary and one highlight. The worker makes one OpenAI image edit and returns one WebP asset resampled to the source dimensions. The 60–30–10 proportions are prompt guidance over suitable editable surfaces, not a measured or guaranteed pixel distribution. Skin, faces, logos, text and product identity may still change; inspect the result before use.

Images are limited to 4 megapixels and aspect ratio 1:3 to 3:1. Save the Idempotency-Key before submitting. An uncertain accepted edit is not automatically sent again, and a failed job records any returned model usage. Existing keys with image.extend permission can use the route; new keys include image.brand. Check job.brandColors, job.warnings and /v1/runs. Locally, run npm run image-api:worker to process queued jobs.

# Upload and complete input.jpg using section 02, then set its ready asset ID.
export IMAGE_API_ASSET_ID="asset_0123456789abcdef0123456789abcdef"
BRAND_KEY="brand-$(uuidgen)" # Save this key with the job ID.
curl --fail-with-body -sS "$IMAGE_API_URL/v1/images/brand-colors" \
  -X POST -H "Authorization: Bearer $IMAGE_API_KEY" \
  -H "Content-Type: application/json" -H "Idempotency-Key: $BRAND_KEY" \
  -d '{"input":{"assetId":"'"$IMAGE_API_ASSET_ID"'"},"options":{"neutrals":["#F3F1EB","#23272B"],"primary":"#1A365D","highlight":"#00C9B7"}}' \
  -o brand-job.json

# Poll GET /v1/jobs/{jobId} with the same bearer key until status=succeeded.
# Find outputs[].assetId where role=image, GET /v1/assets/{assetId},
# then fetch its downloadUrl. Local transfer URLs require the bearer key.
# Reuse BRAND_KEY after a lost submit response; a new key can incur a new edit.

Process for web: assemble responsive layouts

POST /v1/images/process-for-web creates one asynchronous image.processForWeb job from an uploaded asset. Supply 1–4 formats or omit options for landscape (1200 × 675), square (900 × 900), portrait (720 × 900) and mobile (540 × 960). Standard formats offer three bottom-aligned horizontal placements and, for a clear, complete-touching or minor-cutoff bottom edge, six centre/top placements. Mobile has one horizontally centred placement in the lower half, plus centre and top when the bottom edge is complete. Its subject occupies at most about 30% of frame height, with padding around it. An optional nonempty backgroundHint selects FLUX Fill Pro for the shared expansion; otherwise FLUX High is used.

The worker checks the original subject bottom edge, expands one master image, extracts a candidate subject, then evaluates whether that extraction is clean. Read processForWeb.mode after the job succeeds. A layers result provides an opaque background and a tightly trimmed transparent cutout for each ready variant. Place the cutout at subjectPlacement.x and subjectPlacement.y on the background at its native width and height; all four values are integer pixels in that variant’s canvas. A flat result provides one finished crop image per ready variant.

Success always includes an expandedScene asset. Iterate processForWeb.variants, use placement and verticalPlacement to select a position, skip entries with status: "unavailable", and match job.outputs by both role and variantId. Variant asset IDs also appear directly as backgroundAssetId, subjectAssetId or imageAssetId. Inspect evaluation.verdict and evaluation.reason when deciding whether to publish; generated backgrounds and anatomy still need visual review. The taller shared master used for mobile framing can raise the expansion cost.

// Node 20+; first upload an image as shown in section 02.
// Set IMAGE_API_ASSET_ID to the ready asset ID. Persist IMAGE_API_JOB_KEY
// before the first POST, and reuse it after a lost response.
import { mkdir, writeFile } from "node:fs/promises";

const base = new URL(process.env.IMAGE_API_URL ?? "https://img-analyse.vercel.app");
const key = process.env.IMAGE_API_KEY;
const assetId = process.env.IMAGE_API_ASSET_ID;
const jobKey = process.env.IMAGE_API_JOB_KEY;
if (!key || !assetId || !jobKey) throw new Error("Set IMAGE_API_KEY, IMAGE_API_ASSET_ID and IMAGE_API_JOB_KEY");
if (!/^[A-Za-z0-9:._-]{8,128}$/.test(jobKey)) throw new Error("IMAGE_API_JOB_KEY must be 8–128 URL-safe characters");

async function api(path, init = {}) {
  const response = await fetch(new URL(path, base), {
    ...init, headers: { ...init.headers, Authorization: "Bearer " + key },
  });
  const data = await response.json();
  if (!response.ok) throw new Error(response.status + ": " + JSON.stringify(data.error));
  return data;
}

const accepted = await api("/v1/images/process-for-web", {
  method: "POST",
  headers: { "Content-Type": "application/json", "Idempotency-Key": jobKey },
  body: JSON.stringify({ input: { assetId }, options: {
    formats: [{ id: "landscape", width: 1200, height: 675 },
              { id: "square", width: 900, height: 900 },
              { id: "mobile", width: 540, height: 960 }],
  } }),
});
console.log("Job ID (save this):", accepted.id);

let job = accepted;
while (!["succeeded", "failed", "cancelled", "reconciling"].includes(job.status)) {
  await new Promise(resolve => setTimeout(resolve, 3000));
  job = await api("/v1/jobs/" + accepted.id);
}
if (job.status !== "succeeded") throw new Error("Job " + job.status + ": " + JSON.stringify(job.error));

async function saveAsset(id, filename) {
  const asset = await api("/v1/assets/" + id);
  if (!asset.downloadUrl) throw new Error("Asset is not ready: " + id);
  const url = new URL(asset.downloadUrl);
  // Local transfer routes need the bearer key. Do not send it to cloud signed URLs.
  const local = url.origin === base.origin && url.pathname.startsWith("/v1/local-objects/");
  const response = await fetch(url, { headers: local ? { Authorization: "Bearer " + key } : {} });
  if (!response.ok) throw new Error("Download failed: " + id + " (" + response.status + ")");
  await writeFile(filename, Buffer.from(await response.arrayBuffer()));
}

await mkdir("web-layouts", { recursive: true });
const manifest = { jobId: job.id, mode: job.processForWeb.mode, evaluation: job.processForWeb.evaluation, sourceAnalysis: job.sourceAnalysis ?? null, variants: [] };
for (const variant of job.processForWeb.variants) {
  if (variant.status !== "ready") {
    manifest.variants.push({ id: variant.id, formatId: variant.formatId, placement: variant.placement, verticalPlacement: variant.verticalPlacement, status: variant.status, reason: variant.reason });
    continue;
  }
  if (job.processForWeb.mode === "layers") {
    if (!variant.backgroundAssetId || !variant.subjectAssetId || !variant.subjectPlacement) throw new Error("Missing layer metadata: " + variant.id);
    const backgroundFile = "web-layouts/" + variant.id + "-background.webp";
    const subjectFile = "web-layouts/" + variant.id + "-subject.webp";
    await saveAsset(variant.backgroundAssetId, backgroundFile);
    await saveAsset(variant.subjectAssetId, subjectFile);
    manifest.variants.push({ id: variant.id, formatId: variant.formatId, placement: variant.placement, verticalPlacement: variant.verticalPlacement,
      width: variant.width, height: variant.height, status: "ready", subjectPlacement: variant.subjectPlacement,
      backgroundAssetId: variant.backgroundAssetId, subjectAssetId: variant.subjectAssetId, backgroundFile, subjectFile });
  } else {
    if (!variant.imageAssetId) throw new Error("Missing finished image: " + variant.id);
    const imageFile = "web-layouts/" + variant.id + ".webp";
    await saveAsset(variant.imageAssetId, imageFile);
    manifest.variants.push({ id: variant.id, formatId: variant.formatId, placement: variant.placement, verticalPlacement: variant.verticalPlacement,
      width: variant.width, height: variant.height, status: "ready", imageAssetId: variant.imageAssetId, imageFile });
  }
}
await writeFile("web-layouts/manifest.json", JSON.stringify(manifest, null, 2));

Video: select usable stills locally

POST /v1/videos/process queues video.frames and returns HTTP 202 before decoding or AI work. Upload first through POST /v1/video-assets, PUT the returned bytes, then complete via /v1/assets/{assetId}/complete. SDK: client.uploadVideo(bytes, contentType, uploadKey), then client.processVideo(assetId, options, jobKey, signal, webhook).

{
  "input": {
    "assetId": "asset_0123456789abcdef0123456789abcdef"
  },
  "options": {
    "maxFrames": 10,
    "reviewLimit": 30,
    "sampleFps": 2,
    "minSharpness": 35
  }
}

The local FFmpeg worker samples low-resolution previews, discards blank/poor exposure, blur and nearly identical frames, and chooses sharp candidates spread across time. AI reviews only that shortlist in batches of five for sharp subject detail, visibility, pose and framing. Major or uncertain top/side cutoffs are rejected; bottom cutoffs are allowed. Up to ten distinct approved frames are extracted at original dimensions as WebP images. If fewer pass, fewer are returned; no weak frames are added to fill the list.

Limits: 200 MiB, three minutes, 4K, square pixels and 240 fps. Sampling allows 0.5–4 fps; AI review is capped at 40 candidates, default 30. The sharpness threshold is a heuristic and can be adjusted for low-texture footage. This currently requires local storage and FFmpeg/FFprobe; hosted mode is unavailable. No fal generation, cleanup or RunPod inference is used. Video text remains in the stills.

Use tenant/request webhooks for validating, sampling, filtering, reviewing and extracting progress and completion. videoProcessing reports sample/discard/review counts; its final frames include timestamps, asset IDs, dimensions and per-frame decisions. Outputs use role frame. Existing image-analysis keys work. An uncertain AI batch is not automatically repeated; GET the saved job for recovery.

Process IG post: extract a subject or return the original with background guidance

POST /v1/images/process-ig-post queues instagram.processPost and returns HTTP 202 before provider work. Upload and complete an asset first. Use tenant or request webhooks for stage/completion events, or recover the saved job through GET. SDK: client.processIgPost(assetId, options, key, signal, webhook).

Original screening checks extraction suitability, overlay text, subject bounds, cutoff edges and background composition. A distinct subject or coherent interacting group can proceed. Posters, collages, unusable scenes, distributed flat-lays without a standout subject, or unknown extraction composition return the original. A dominant single dish or product can qualify even in an overhead photo.

Extraction-ineligible is a successful result: status: succeeded, processIgPost.outcome: original_returned, suitability.eligible: false, an explanation and only the uploaded image asset. Inspect suitability.extractionComposition, reasons, summary and screeningFeedback. No cleanup, repair, extraction, cutout/mask or final cutout evaluation runs. Existing text and graphics remain. A failed screening request is a separate error, not this successful branch.

Suitable originals use conditional overlay cleanup and cleaned-source feedback, optional minor-cutoff repair, one full-frame RunPod extraction, and advisory quality evaluation on light/dark mattes. Repair defaults to OpenAI; FLUX is optional. Complete/touching subjects skip repair; major/unknown edges skip speculative repair with a warning. Processed success returns aligned image, cutout and mask. The image retains its original background. Evaluation pass/fail/unknown does not discard available outputs; actual processing failures can fail the job.

{
  "input": {
    "assetId": "asset_0123456789abcdef0123456789abcdef"
  },
  "options": {
    "extractText": false,
    "expansionProvider": "openai",
    "expansionFraction": 0.12,
    "mobileViewport": {
      "width": 360,
      "height": 640
    }
  }
}

Choose whether to extract a subject

options.extractSubject defaults to true. Set false for original screening and background guidance only: the endpoint returns the original image with interests, primary selection and crop suggestions, skipping text cleanup, repair, extraction and final cutout evaluation. Existing overlay text stays; text-extraction and repair options have no effect. The test UI has an Extract subject when eligible checkbox.

processIgPost.extractionRequested records the choice. Original-returned results use originalReturnReason: extraction_not_requested for consumer opt-out, or unsuitable_source for rejection by the extraction gate. Eligibility remains a separate image assessment and can be true when extraction was disabled. Inspect outcome and actual output roles before expecting cutout/mask assets. Explicit true and omitted are equivalent for replay; a mode change needs a new idempotency key.

Use an extraction-ineligible image as a background

A multi-dish restaurant scene can be unsuitable for isolation but useful intact on a website. The same initial screening returns processIgPost.backgroundUse independently of extraction eligibility: background suitability/summary/confidence, broad areasOfInterest, dontCropAreas, dontOverlapAreas, textSafeAreas, focalPoint and primaryAreaOfInterest. This works even when sourceAnalysis is absent. The older source-analysis interests are limited to faces and held objects.

primaryAreaOfInterest identifies an area index, kind, label, normalized bounds, confidence and rationale. Selection weighs central/rule-of-thirds placement, prominence, sharpness, contrast, colour, lighting and context. It is null when no priority is defensible. A primary area for cropping does not make the scene eligible for extraction.

Crop positioning keeps the primary region whole when possible, then favours complete neighbours. A wide crop can move upward to include a primary soup bowl and trim a secondary lower plate. It is not a fixed centre band. primaryAreaPreserved reports full retention in the safe crop, or the cover alternative when the safe crop is null.

• Mobile 100% height: 9:16.
• Mobile 50% height: 9:8.
• Mobile 30% height: 9:4.8, equivalent to 15:8.
• Square 1:1 and desktop 16:9 / 21:9 remain available.

The default viewport is 360 × 640 CSS pixels; options.mobileViewport changes the three mobile ratios. Each crop has an ID, aspect ratio, status, normalized crop, CSS object-position percentages and reason. suggested preserves primary and don’t-crop regions. review_needed means uncertain framing. contain_recommended has null crop/objectPosition; show the whole image or explicitly review coverCrop, which may clip secondary protected content.

All background boxes and crop coordinates refer to the original oriented input with sourceSize. They are not output-pixel guides. Apply them to the original image, remap overlay regions through the chosen crop, and avoid don’t-overlap areas. Empty text-safe areas mean no reliable quiet space was found. Text-tone and scrim advice does not establish accessible contrast.

The test UI draws interests in orange and the primary in blue, with confidence and selection rationale. Don’t-crop and don’t-overlap toggles use teal and purple. Background crops shows selectable thumbnails and a larger view; 30% height defaults to the wide cover alternative with a warning when needed. Switching crops makes no AI calls and does not create image assets. Background guidance shares original screening; only geometry and previews are computed locally.

For processed outputs, sourcePlacement maps pre-repair source regions into output pixels. processIgPost.guides adds visible subject bounds, a title baseline 3% below the subject top and padded face/held-object regions. Repair allows 2–20%, up to 4096 px per side and 4 megapixels; FLUX also limits sides to 2560 px. There is no automatic support recovery or background reconstruction.

Older results may omit newer metadata. Reloading or reusing an idempotency key retrieves that saved outcome and viewport; use a new key for an intentional new screening attempt. Claude malformed JSON/schema responses get one persisted retry per screening phase. Extra background object properties are discarded; required findings and coordinates remain validated. Exhausted screening failures retain usage and need a new run rather than a saved-result reload.

Instagram post to responsive web layers

POST /v1/images/instagram-process-for-web takes any uploaded image accepted by Process for web. An initial OpenAI vision screening checks for text added over the photo, scene quality and image attributes in one call when no overlay text is found. Lettering photographed on tattoos, billboards, or street signs does not count as an overlay. instagramPost.textDetection.status is present, absent or uncertain. Only absent skips text cleanup; instagramPost.layerization.status is then skipped. options.extractText defaults to false: present or uncertain uses fal text removal ($0.04/image), leaves extractedText empty, and screens the cleaned image again. Set extractText: true to use Ideogram V3 Layerize Text ($0.09/image) and return raw words in instagramPost.layerization.extractedText. Removal-only can also erase photographed writing such as tattoos and signs, so inspect the cleaned image. The cleaned-image screening returns image attributes when text was removed. The worker reuses screened attributes without a separate OpenAI attribute call; older saved jobs fall back to that call. The text-free image is an internal processing input. Screening rejects with HTTP 422 INSTAGRAM_POST_UNSUITABLE for genuine inset photos or collage, unusable visual quality, or damage to the subject. The amount of text is not a rejection reason. A carousel arrow is not an inset image.

options.extractShadow defaults to false, leaving the generated soft contact shadow out of the subject cutout. Set it to true to add that shadow when surface contact is detected; responsiveLayers.subject.contactShadow reports none or soft. This is local processing and adds no provider call. Existing haze in the model cutout may remain. options.mobileOnly defaults to true, including when options is omitted. A successful default job returns exactly two WebP assets: one 9:24 mobile background and one transparent cutout. The visible subject, excluding any soft contact shadow, fills about 90% of the mobile background width; draw the cutout at responsiveLayers.mobile.subjectPlacement without resizing. The output crop and subject retain the expanded master’s native pixel scale. responsiveLayers.mobile.titleGuide.lineY is a title baseline 3% of the visible subject height below its top; draw the title on this line behind the cutout. responsiveLayers.mobile.areasOfInterest.regions maps individual sourceAnalysis.areasOfInterest face and held-object boxes into crop-local pixels with 15% padding per side. Held phones, trophies, signs, cups and instruments are included when visible; unrelated background objects are excluded. The test UI draws separate orange rectangles. Detection reuses the existing vision call. Older saved analysis falls back to its combined protected focus. For compatibility, responsiveLayers.mobile.dontOverlap maps the protected sourceAnalysis.attributes.neverCropArea from the oriented input into crop pixels, then adds 15% of the rectangle’s width and height on every side. Its topY and bottomY mark the padded bounds, without a subject-height cap; it reports unavailable when no protected area was identified. When a reliable floor/wall edge is detected, responsiveLayers.mobile.boundaryPoints gives its crop-local polyline, including any incline. The floor line is optional. These guides add no image-generation call. Set options.mobileOnly: false to request desktop and 9:16 mobile backgrounds with one shared cutout; reliable support boundaries may also produce farBackground and supportSurface split assets in that mode. The mobile-only default omits all split assets. When extractShadow is true and the subject rests on a surface, the subject cutout can include an approximate soft contact shadow. The cleaned image and expanded master are processing intermediates. A subject that cannot be separated cleanly fails with INSTAGRAM_SUBJECT_UNLAYERABLE.

The marker coordinates are pixels in the mobile background. They are preview guides, not pixels drawn into either output asset. Areas of interest can be unavailable if image attributes find no usable focus; the floor edge can be absent. The local test preview shows the title and padded rectangles by default.

The initial screening is an OpenAI vision call. Posts without overlay text avoid a fal cleanup charge. Posts with visible or uncertain overlays pay $0.04 for removal only, or $0.09 for extraction with Ideogram, plus a second OpenAI screening call. An early rejection avoids fal text cleanup. Its fal request is saved under the Idempotency-Key; retry the same request and key after a pending response to resume collection without another fal submission. A lost submit receipt is treated as uncertain and is not automatically resubmitted. Poll the accepted job ID, then fetch the base and any split asset IDs with GET /v1/assets/{assetId} and follow each downloadUrl. The cleaned image and expanded master are processing intermediates and are not returned as extra output images. Rejected screening appears in /v1/runs. Existing keys need image.layouts, scene.separate and text.remove permissions.

// Node 20+; upload an image and persist IMAGE_API_JOB_KEY.
const base = process.env.IMAGE_API_URL ?? "http://127.0.0.1:4114";
const key = process.env.IMAGE_API_KEY;
const assetId = process.env.IMAGE_API_ASSET_ID;
const jobKey = process.env.IMAGE_API_JOB_KEY;
if (!key || !assetId || !jobKey) throw new Error("Set IMAGE_API_KEY, IMAGE_API_ASSET_ID and IMAGE_API_JOB_KEY");
const response = await fetch(new URL("/v1/images/instagram-process-for-web", base), {
  method: "POST",
  headers: { Authorization: "Bearer " + key, "Content-Type": "application/json", "Idempotency-Key": jobKey },
  body: JSON.stringify({ input: { assetId } }), // Mobile-only and removal-only are the defaults.
});
const body = await response.json();
if (response.status === 422) {
  console.log("Post rejected:", body.error.code, body.error.message);
} else if (!response.ok) {
  throw new Error(response.status + ": " + JSON.stringify(body.error));
} else {
  console.log("Accepted job:", body.id, "text detection:", body.instagramPost.textDetection?.status, "Text cleanup:", body.instagramPost.layerization.status, "extracted text:", body.instagramPost.layerization.extractedText);
  let job = body;
  while (!["succeeded", "failed", "cancelled"].includes(job.status)) {
    await new Promise(resolve => setTimeout(resolve, 3000));
    const polled = await fetch(new URL("/v1/jobs/" + body.id, base), { headers: { Authorization: "Bearer " + key } });
    job = await polled.json();
    if (!polled.ok) throw new Error(JSON.stringify(job.error));
  }
  if (job.status !== "succeeded") throw new Error(JSON.stringify(job.error));
  const layers = job.processForWeb.responsiveLayers;
  console.log("Mobile background:", layers.mobile.backgroundAssetId, layers.mobile.subjectPlacement);
  console.log("One subject layer:", layers.subject.assetId, layers.subject.contactShadow);
  console.log("Title line:", layers.mobile.titleGuide);
  console.log("Protected area lines:", layers.mobile.dontOverlap);
  console.log("Optional floor edge:", layers.mobile.boundaryPoints);
  // GET /v1/assets/{assetId} for each required ID, then fetch its downloadUrl.
  // Draw the subject at subjectPlacement over the background at native scale (1:1).
}