Runway

Generate a polished creative product ad video from a product URL or product image plus a campaign idea. CREDIT MODE: Runway account availability is checked against the currently connected workspace. Use this for product marketing / creative ad requests like: - "Make a jewelry ad featuring a chameleon in a jewelry store" - "Turn this product page into a cinematic ad" - "Use this product image and this character reference to make a funny product commercial" v1 supports only format: `creative_ad_video`. Workflow: 1. Get the exact product image from `productImages[0]` or extract the best high-resolution product image from `productUrl`. 2. Use optional `referenceImages` as character, scene, mood, or world references. 3. Internally generate a 3x3 storyboard image, preferring GPT Image 2 for paid workspaces and falling back to Nano Banana Pro otherwise. 4. Use that storyboard as sequential shot guidance for Seedance 2, along with the product image and references, to create the final video. PRODUCT URL FALLBACK: - `productUrl` extraction is best-effort. Some ecommerce sites block automated access or hide product images behind scripts. - If extraction fails, do not retry the same URL or claim Runway cannot make the ad. Ask the user for a screenshot of the product page or a direct product image upload, then call this tool again with `productImages` / `productImageFile`. - Do not call `generate_image` with a text description of the product to create the product reference. That produces a lookalike, not the real SKU. - For branded/luxury products, the exact product image is required for an accurate ad. Only invent or generate a stand-in product if the user explicitly agrees that an approximate substitute is acceptable. Inputs: - productUrl: Product landing page URL. The tool will scrape metadata and rank image candidates, preferring high-resolution product/gallery images. - productImages: Product image URLs from uploads, recent assets, or previous generations. Preferred over productUrl extraction. - productImageFile: ChatGPT-only uploaded product image file. Use this when ChatGPT provides a user-uploaded product image. - referenceImages: Optional character/scene/mood/world references. - promptText: Required creative campaign idea. If the user requests a named artist/director/photographer/studio style, describe the visual qualities instead and do not include the name. - duration: 10 by default; use 15 only when the user asks for a longer ad. STYLE SAFETY: Do not include names of artists, directors, photographers, studios, or other creators anywhere in prompts submitted to Runway tasks. When style direction is needed, translate it into camera language, palette, lighting, genre, era, composition, and material texture. USER-FACING REPLIES: DO NOT mention internal model names unless asked. Do not mention the intermediate storyboard unless useful. Tell the user the creative ad video is generating and the viewer will update when ready.

generate_product_marketing_video

Generate a sound effect from a text description — foley, ambience, impacts, UI sounds, creature noises, whooshes, loops — and return an MP3. Describe the sound source, texture, environment, and how it evolves in `promptText`. CREDIT MODE: Runway account availability is checked against the currently connected workspace. COST: 1 credit per second of requested `duration` (rounded up); 2 credits when `duration` is omitted and the model picks the length. Durations run 0.5–30 seconds. WHEN TO USE: - "a door creaking open", "rain on a tin roof, 10 seconds", "sci-fi laser blast", "a looping crackling campfire", "footsteps on gravel", "a notification chime". - NOT for music (use `generate_music`), NOT for speech or voiceover (use `generate_speech`), NOT for adding sound to an existing video (generate the effect here, then combine externally). AVAILABILITY: sound effects are behind a workspace feature flag. If the API rejects the task type, tell the user the feature is not enabled on their workspace and stop — do not retry. PROMPTING TIPS: name the source first ("heavy wooden door"), then the action ("slams shut"), then the space ("in a stone hallway, long reverb tail"). Mention pacing for longer effects ("starts quiet, builds to a roar"). Set `loop: true` for ambience or beds meant to repeat seamlessly. Max 450 characters. USER-FACING REPLIES: Talk about the sound (what it is, how long, what it's for), not the model or task machinery. Do not mention ElevenLabs unless the user asks. The output is an MP3 the viewer plays inline; headless agents get the URL from `get_task`. Parameters: - promptText: Required. What the sound effect should be. Max 450 characters. - duration: Optional. Length in seconds, 0.5–30. Omit to let the model choose. - loop: Optional. true to design the effect to loop seamlessly (ambience, beds). Default false. - name: Optional. Display name for the asset in the user's Runway library. Defaults to the prompt.

generate_sound_effect

Turn text into spoken audio (text-to-speech) with one of Runway's preset voices and return an MP3. Pass the exact words to speak in `text` and pick a `voice` by name. CREDIT MODE: Runway account availability is checked against the currently connected workspace. COST: 1 credit per 50 characters of `text`, rounded up (minimum 1). Max 5000 characters per call — split longer scripts into several calls. MODELS (pick with `model`; default `eleven_v3`): - `eleven_v3`: Most expressive. Understands inline delivery tags in the text such as [whispers], [laughs], [sighs], [excited]. Best for dialogue, characters, and emotive narration. - `eleven_multilingual_v2`: Steadier and more predictable; ignores delivery tags. Best for long neutral narration, product voiceover, and non-English text. VOICES (pick with `voice`; default `Leslie`). Choose by the user's description ("deep male narrator", "cute cartoon voice", "British woman, calm"); do not ask the user to pick from the full list unless they want to browse: - Maya: feminine, young, african american accent, chill (conversational) - Arjun: masculine, middle aged, indian accent, intense (narrator) - Serene: feminine, young, american accent, calm (narrator) - Bernard: masculine, middle aged, american accent, confident (animated) - Billy: masculine, middle aged, american accent, hyped (animated) - Mark: masculine, middle aged, american accent, excited (animated) - Clint: masculine, middle aged, american accent, husky (animated) - Mabel: feminine, middle aged, us southern accent, relaxed (narrator) - Chad: masculine, young, american accent, excited (animated) - Leslie: feminine, middle aged, american accent, professional (narrator) - Eleanor: feminine, middle aged, canadian accent, classy (narrator) - Elias: masculine, old, american accent, calm (narrator) - Elliot: masculine, middle aged, american accent, anxious (animated) - Grungle: masculine, middle aged, american accent, raspy (animated) - Brodie: masculine, young, american accent, chill (animated) - Sandra: feminine, middle aged, american accent, sassy (conversational) - Kirk: masculine, middle aged, british accent, rough (animated) - Kylie: feminine, young, australian accent, confident (influencer) - Lara: feminine, young, american accent, confident (influencer) - Lisa: feminine, young, german accent, casual (conversational) - Malachi: masculine, middle aged, american accent, confident (animated) - Marlene: feminine, middle aged, british accent, classy (narrator) - Martin: masculine, middle aged, american accent, rough (animated) - Miriam: feminine, middle aged, british accent, calm (narrator) - Monster: masculine, old, american accent, deep (animated) - Paula: feminine, middle aged, british accent, calm (conversational) - Pip: feminine, young, american accent, cute (animated) - Rusty: masculine, old, british accent, raspy (animated) - Ragnar: masculine, middle aged, american accent, deep (animated) - Xylar: neutral, middle aged, american accent, modulated (animated) - Maggie: feminine, young, american accent, calm (narration) - Jack: masculine, old, irish accent, sailor (video games) - Katie: feminine, young, american accent, soft (news) - Noah: masculine, young, american accent, calm (meditation) - James: masculine, middle aged, australian accent, casual (conversational) - Rina: feminine, young, american accent, calm (meditation) - Ella: feminine, young, american accent, emotional (narration) - Frank: masculine, young, american accent, deep (narration) - Claudia: feminine, young, british-swedish accent, seductive (characters) - Niki: feminine, middle aged, american accent, friendly (narration) - Vincent: masculine, old, australian accent, calm (news) - Tom: masculine, middle aged, british accent, authoritative (news) - Wanda: feminine, middle aged, american accent, pleasant (interactive) - Benjamin: masculine, middle aged, american accent, deep (narration) - Kiana: feminine, young, american accent, whisper (audiobook) - Rachel: feminine, young, english-swedish accent, childish (animation) WHEN TO USE: - "read this aloud", "voiceover for my video: …", "narrate this script in a British accent", "say 'welcome back' in a robotic voice". - NOT for music (use `generate_music`), NOT for sound effects (use `generate_sound_effect`), NOT for cloning a specific person's voice, NOT for translating or re-voicing an existing audio/video file. PROMPTING TIPS: `text` is spoken verbatim — do not include stage directions as plain prose. With `eleven_v3` you may add bracketed delivery tags inline, e.g. "[whispers] Don't move. [pause] Did you hear that?"; `eleven_multilingual_v2` ignores tags. Use punctuation and line breaks to control pacing. Set `languageCode` when the text is not English and pronunciation matters. USER-FACING REPLIES: Talk about the voiceover (which voice, what it says, how long), not the model or task machinery. Do not mention ElevenLabs unless the user asks. The output is an MP3 the viewer plays inline; headless agents get the URL from `get_task`. Parameters: - text: Required. The words to speak, verbatim. Max 5000 characters. - voice: Optional. Preset voice name from the list above. Default `Leslie`. - model: Optional. `eleven_v3` (default, expressive, supports tags) or `eleven_multilingual_v2` (steady). - speed: Optional. 0.7–1.2; 1.0 is normal. - languageCode: Optional. ISO 639-1 code (e.g. `es`, `de`, `pt-br`) to enforce pronunciation for non-English text. - name: Optional. Display name for the asset in the user's Runway library. Defaults to the text.

generate_speech

Generate OR edit a video using a Runway-hosted video model. Pass the source video as `referenceVideo` to edit/restyle it. CREDIT MODE: Runway account availability is checked against the currently connected workspace. PAID PLAN: If this tool returns paid_plan_required, follow the NEXT line in that result — it asks for `show_plans_and_credits` with the blocked action as `gatedCapability`, which renders the upgrade card. Do not retry with a different video model from this schema; every video model needs a paid plan. Only offer models `whoami` lists in `availableVideoModels` / `availableImageModels`, and only after calling `whoami`. CHOOSING BETWEEN THIS TOOL AND `edit_video` FOR FOOTAGE EDITS: - Use `edit_video` (Aleph 2.0 in-context editing) when the user wants a targeted, surgical change to existing footage and everything else must stay exactly as shot: "make the sneakers red", "change the outfit", "remove the object on the wall", "change the season to winter", "relight the scene", element swaps, or restyles that must preserve the original motion/framing/timing — or whenever the user mentions Aleph or Edit Studio. `edit_video` also requires a paid workspace — do not send free-plan users there as a workaround. - Use THIS tool's video-to-video mode (`referenceVideo`, seedance-2) when the source video is more of a loose reference: motion transfer, style/structure inspiration, heavy re-imagining, or when the source violates Aleph's constraints (must be 2-30s, <=1080p, <=30fps, <=10 cuts, conventional aspect ratio). - The two are mutual fallbacks: if an `edit_video` result was unsatisfying (or the source is out of Aleph's constraints), try v2v here; if a v2v edit here changed more than the user asked for, suggest `edit_video` instead. For creative product ad videos from a product URL or product image, use `generate_product_marketing_video` instead. It handles product-image extraction, cinematic storyboard generation, and the final ad video. For multi-shot videos (one prompt drives 3-5 connected scenes), use `generate_multishot_video` instead. MODES: 1. Text-to-video — pass `promptText` only. Most models support this directly. `gen-4-turbo` is the exception: for best reliability, generate a still image first with `generate_image`, then animate that image with `gen-4-turbo`. Example: "aerial drone shot, slow push forward over misty mountains at sunrise". 2. Image-to-video — pass `startFrame.url` (or ChatGPT's `startImageFile`) plus `promptText` to animate a still image. Use this when you want to bring a specific image to life. Example: a product photo + "slow 360 orbit around the bottle". 3. Video-to-video (edit/restyle) — pass `referenceVideo.url` plus a `promptText` describing the desired change. Use this to *modify* an existing video while preserving motion and composition: remove or replace backgrounds, remove unwanted objects/people, add new objects, swap backgrounds, change lighting, restyle scene, alter weather/time-of-day. Supported by the Seedance family (`seedance-2` is the DEFAULT for v2v; `seedance-2.5` handles sources longer than 15s), `kling-o3-pro`, and the WAN family (`wan3`, `wan3-prime`). Example: an existing dance video + "remove the background and place the dancer on a clean white studio backdrop". VIDEO EDITING REQUESTS: - If the user asks to remove a background, remove an object/person, clean up a scene, add something to an existing video, or otherwise edit footage, DO NOT say Runway MCP cannot do it. First consider `edit_video` (Aleph 2.0, paid workspace) per the routing guidance above; otherwise use video-to-video here: pass the source clip as `referenceVideo` / `referenceVideoFile`, default to `seedance-2`, and describe the edit in `promptText`. - For background removal, ask for or use the input video as `referenceVideo` and prompt for the desired replacement/background treatment, e.g. "remove the background and isolate the subject on a plain neutral studio backdrop" or "replace the background with a clean white seamless studio". DURATION DEFAULTS (the tool sets these automatically when you OMIT `duration`): - Text-to-video & image-to-video → 10s (or the model's largest supported value <= 10s; veo-3.1 → 8s). - Video-to-video WITH `referenceVideo.durationSeconds` → largest model-supported value <= source. (e.g. 20s source with kling-o3-pro {5,10,15} → 15s; 8s source with seedance-2 {4..15} → 8s.) - Video-to-video WITHOUT `referenceVideo.durationSeconds` → model's MAX supported duration (15s for seedance/kling) so the edit doesn't get accidentally truncated. DURATION FOR V2V — IMPORTANT: Always pass `referenceVideo.durationSeconds` when you know it (read it from the file metadata you uploaded). It's the only way the tool can know how much of the source to edit. - DO NOT pass `duration: 5` for a 20-second source video — that will silently clip the edit to the first 5 seconds. Just OMIT `duration` and pass `referenceVideo.durationSeconds` instead, and the tool will pick the right output length. - Only set `duration` explicitly when the user asks for a shorter clip ("just make me a 5-second teaser of this 20-second video"). DO NOT use this tool to edit a still **image** → use `generate_image` and pass the image as `referenceImages[0]`. ANIMATE BUTTON (viewer → chat): when a user message says "Animate this image" and names a Runway task ID, the user clicked Animate on a finished image in the inline viewer. Follow this two-step flow: 1. Do NOT generate yet. Reply in one or two sentences: confirm you can turn that image into a short video (default 10 seconds), describe the image briefly if you know its prompt, and ask what should happen in it — motion, camera, mood — and whether they want a different length. 2. Once they answer, call this tool with `startFrame.taskId` set to that task ID (never retype the image URL), `promptText` built from their answer, and `duration` only if they asked for a specific length. Keep the ratio matching the source image when you know it. HANDLING USER ATTACHMENTS: - ChatGPT: if it supplies an uploaded image as `startImageFile`, pass it directly. This tool will download the temporary ChatGPT file server-side, upload it to Runway, and use it as `startFrame.url`. - ChatGPT: if it supplies an uploaded video as `referenceVideoFile`, pass it directly for video-to-video edits. This tool will download the temporary ChatGPT file server-side, upload it to Runway, and use it as `referenceVideo.url`. - ChatGPT: NEVER pass local paths such as `/mnt/data/file.png` or `/mnt/data/file.mp4` as `startFrame.url` or `referenceVideo.url`; the remote MCP server cannot read them. Use `startImageFile` / `referenceVideoFile` instead. - Claude/Cursor/local agents: if the user attached a local image or video, run `init_upload` -> curl -> `complete_upload` first to get a Runway-hosted asset URL, then pass it as `startFrame.url`, `endFrame.url`, `referenceImages[].url`, or `referenceVideo.url`. REUSING ASSETS: URLs returned by other Runway tools (image outputs from `generate_image`, the asset URL from `complete_upload`) are stable, hosted asset URLs. Pass them directly into the relevant field — never re-upload. COPY THESE URLS VERBATIM. They end in a long `_jwt=` authorization token that is meaningless if even one character changes. Copy the whole string exactly as it appeared in the earlier tool result; never retype it, truncate it, shorten it for readability, or reconstruct it from the parts you remember. If you cannot reproduce the URL exactly, call `init_upload` and use the URL it returns instead. Note: outputs of `generate_video` itself are NOT yet usable as `referenceVideo` (re-upload via `init_upload` if needed). USER-FACING REPLIES: Pick the right model internally, but DO NOT mention the model name to the user (e.g. "using Veo 3.1", "with Seedance 2") unless they explicitly ask which model was used. Talk about the video content — subject, motion, mood — not the engine. MODEL NAMES (a user may name a model by its product name; pass the id on the left): - `seedance-2` = Seedance 2.0 - `seedance-2.5` = Seedance 2.5 - `seedance-2-fast` = Seedance 2.0 Fast - `seedance-2-mini` = Seedance 2.0 Mini - `kling-o3-pro` = Kling O3 Pro - `kling-3-pro` = Kling 3.0 Pro - `gen-4.5` = Gen-4.5 - `veo-3.1` = Veo 3.1 - `grok-imagine-1.5` = Grok Imagine 1.5 - `gemini-omni-flash` = Gemini Omni Flash - `hailuo-3` = MiniMax Hailuo 3.0 - `wan3` = WAN 3.0 - `wan3-prime` = WAN 3.0 Prime - `gen-4-turbo` = Gen-4 Turbo MODELS (`*` = image-first workflow): | model | tier | t2v | i2v | v2v | end | refs | audio | durations | max res | |-------------------|------|-----|-----|-----|-----|------|-------|------------|---------| | seedance-2 | paid | Y | Y | Y | Y | Y | Y | 4-15 | 4k | | seedance-2.5 | paid | Y | Y | Y | Y | Y | Y | 4-30 | 1080p | | seedance-2-fast | paid | Y | Y | Y | Y | Y | Y | 4-15 | 720p | | seedance-2-mini | paid | Y | Y | Y | Y | Y | Y | 4-15 | 720p | | kling-o3-pro | paid | Y | Y | Y | Y | Y | Y | 5, 10, 15 | 1080p | | kling-3-pro | paid | Y | Y | - | Y | - | Y | 5, 10, 15 | 1080p | | gen-4.5 | paid | Y | Y | - | - | - | - | 2-10 | 1080p | | veo-3.1 | paid | Y | Y | - | Y | - | Y | 4, 6, 8 | 1080p | | grok-imagine-1.5 | paid | Y | Y | - | - | Y | Y | 1-15 | 1080p | | gemini-omni-flash | paid | Y | Y | - | - | Y | - | 3-10 | 720p | | hailuo-3 | paid | Y | Y | Y | Y | Y | Y | 5-15 | 1080p | | wan3 | paid | Y | Y | Y | Y | Y | Y | 2-30 | 1080p | | wan3-prime | paid | Y | Y | Y | Y | Y | Y | 2-30 | 1080p | | gen-4-turbo | paid | * | Y | - | - | - | - | 5, 10 | 720p | Legend: t2v = text-to-video, i2v = image-to-video (startFrame), v2v = video-to-video (referenceVideo, edit/restyle existing video), end = end-frame target, refs = reference images. `*` for gen-4-turbo means it is not native text-to-video: it needs a starting image. For text-only requests, create a still image first, then call gen-4-turbo with that image as `startFrame`. Picking heuristic: - seedance-2 (DEFAULT for t2v/i2v/v2v) — best general-purpose, including existing-footage edits like removing/replacing backgrounds, removing objects/people, and adding new elements. Pick this unless you have a reason not to. The only Seedance tier that does 4k (reserve 4k for explicit 4K requests — it costs ~4x 1080p). - seedance-2.5 — newest Seedance generation. Same capabilities as seedance-2 plus durations up to 30s and prompts up to 15000 characters, and goes up to 1080p (no 4k). Pick when the user asks for Seedance 2.5 by name or wants a clip longer than 15s. Availability is account-gated: if the API rejects it with an unknown-task-type error, fall back to seedance-2. - seedance-2-fast — same capabilities as seedance-2 but faster and cheaper; caps at 720p. Pick when latency/cost matter more than resolution and 1080p isn't needed. - seedance-2-mini — cheapest Seedance tier; identical features, 720p max. Pick for budget-sensitive or high-volume generation. - kling-o3-pro — pick when consistency matters: character/product must look identical across shots, brand or serialized content, or when v2v needs to preserve identity. Best for ads, episodic content, and product storytelling. Optimized for 1-2 main subjects. - kling-3-pro — pick when prompt fidelity matters more than reference-driven consistency: complex multi-character scenes (3+ people), crowded environments, experimental scripts, rapid ideation without reference assets. (No v2v support.) - veo-3.1 — pick when photorealism and natively-generated audio matter (Veo invents dialogue spoken by characters in-frame, plus ambient sound matched to the scene). (No v2v support.) - gen-4.5 — pick for keyframe-driven storytelling (provide a startFrame and the model anchors at timestamp 0). (No v2v support.) - grok-imagine-1.5 — fast generation with native audio that is ALWAYS ON (no toggle — do not pass generateAudio: false) and audio-driven performances. Pick for quick turnaround, short clips (durations go down to 1s in 1s steps), or reference images addressable from the prompt (tag a reference "hero" and write "@hero" in promptText). Constraints: startFrame and referenceImages are mutually exclusive; up to 7 reference images but they cap output at 720p (1080p is t2v/i2v only); extra ratios 3:2 and 2:3. Availability is account-gated: on an unknown-task-type error, fall back to seedance-2. (No v2v support.) - gemini-omni-flash — paid, real t2v/i2v. Pick for physics-realistic motion (gravity, fluids, collisions), knowledge-grounded scenes, or fusing image + text references in one request. Fixed 720p, no audio, ratios 16:9/9:16/auto only, up to 5 reference images. (No v2v support.) - hailuo-3 — MiniMax Hailuo 3.0. Native audio is ALWAYS ON (no toggle — do not pass generateAudio: false). Pick for keyframe-driven or reference-driven clips up to 15s with sound, or when the user asks for Hailuo/MiniMax by name; it also does v2v. Ratio note: with reference images or a reference video it defaults to `adaptive` (the reference's own framing); `adaptive` is rejected for text-only prompts. Prompts up to 6000 characters. Availability is account-gated: on a 403 "This model is not available for your account." fall back to seedance-2. - wan3 — Alibaba WAN 3.0 (standard). Pick when the user asks for WAN/Alibaba by name without saying Prime, or for long clips with sound: durations run 2-30s and native audio is on by default (pass `generateAudio: false` to turn it off). Takes up to 10 reference images, a reference video of at most 15s, and prompts up to 20000 characters; it also does v2v. Ratio note: with reference images or a reference video it defaults to `adaptive` (the reference's own framing). Start/end frames cannot be combined with reference images or a reference video. Availability is account-gated: on an unknown-task-type error, fall back to seedance-2. - wan3-prime — high-speed WAN 3.0. Same controls as wan3 (t2v/i2v/v2v, 2-30s, audio, refs, prompt limit) but faster and ~40% more credits (14/s at 720p vs 10/s). Pick when the user asks for WAN Prime / WAN 3.0 Prime / wan3_prime by name, or wants faster WAN generations. Same flag gating as wan3: on an unknown-task-type error, fall back to wan3 then seedance-2. - gen-4-turbo — image-to-video model. If the user supplied an image, pass it as `startFrame`. For text-only requests, first call `generate_image`, then call `generate_video` with `model: "gen-4-turbo"` and `startFrame.url` set to the generated image URL. Do not keep retrying if the starter-frame step is still pending or fails. (No v2v support.) Parameters: - model: One of `seedance-2`, `seedance-2.5`, `seedance-2-fast`, `seedance-2-mini`, `kling-o3-pro`, `kling-3-pro`, `gen-4.5`, `veo-3.1`, `grok-imagine-1.5`, `gemini-omni-flash`, `hailuo-3`, `wan3`, `wan3-prime`, `gen-4-turbo`. Default for t2v/i2v: `seedance-2`. Default for v2v (when `referenceVideo` is set): `seedance-2`. - promptText: Required. For t2v/i2v, describe the motion/style. For v2v, describe the edit ("remove the background", "remove the person in the left background", "add a floating logo above the table", "make it nighttime", "snow falling", "cyberpunk neon"). Prompts longer than 3500 characters are truncated, except on `seedance-2.5` (up to 15000), `hailuo-3` (up to 6000) and the WAN family (`wan3`, `wan3-prime`, up to 20000). - ratio: Aspect ratio (16:9, 9:16, 1:1, plus 4:3, 3:4, 21:9, 3:2, 2:3 on some models). Ignored for v2v — the model preserves the source video's ratio. - duration: Seconds (model-dependent — see table). USUALLY OMIT — defaults to 10s for t2v/i2v, and to the largest model-supported value <= source for v2v. See DURATION DEFAULTS above. Only set explicitly when the user asks for a non-default length. - startFrame: `{taskId}` or `{url}` — i2v only. Animate this image. Prefer `taskId` (the Runway task ID of a finished image) whenever the image came from a Runway tool; the server resolves the URL. Otherwise `url` must be public HTTPS or a Runway-hosted asset URL. Do NOT use ChatGPT local paths like `/mnt/data/...`; use `startImageFile` for ChatGPT uploads. - startImageFile: ChatGPT-only uploaded image file param `{download_url, file_id, mime_type, file_name}`. Use this instead of `startFrame` only when ChatGPT provides a user-uploaded file. - endFrame: `{url}` — i2v only. Final-frame target. Requires startFrame. (Seedance family, kling-o3-pro, kling-3-pro, veo-3.1, hailuo-3, wan3, wan3-prime.) - referenceImages: Array of `{url, tag}` — i2v style/subject anchors. (Seedance family, kling-o3-pro, grok-imagine-1.5, hailuo-3, wan3, wan3-prime. hailuo-3 takes up to 9 and WAN takes up to 10, both ignore tags; on either they cannot be combined with startFrame/endFrame. For grok-imagine-1.5 the tag is prompt-addressable: tag "hero" -> write "@hero" in promptText.) - referenceVideo: `{url, durationSeconds?}` — v2v source video for editing/restyling existing footage, including removing/replacing backgrounds, removing objects/people, and adding elements. URL must be a Runway-hosted `/datasets/<UUID>.<ext>` URL from `complete_upload`. Do NOT use ChatGPT local paths like `/mnt/data/...`; use `referenceVideoFile` for ChatGPT uploads. Pass `durationSeconds` from your file metadata when known so the model picks an appropriate output length. - referenceVideoFile: ChatGPT-only uploaded video file param `{download_url, file_id, mime_type, file_name}`. Use this instead of `referenceVideo` only when ChatGPT provides a user-uploaded video. - generateAudio: Whether to generate native audio (model-dependent; default true for audio-capable models). For v2v with kling-o3-pro, controls whether to keep the original audio. grok-imagine-1.5 and hailuo-3 have no toggle (audio is always on) — omit this parameter for them. - resolution: Override default resolution (seedance-2: 480p/720p/1080p/4k; seedance-2.5: 480p/720p/1080p; seedance-2-fast & seedance-2-mini: 480p/720p; veo: 720p/1080p — 1080p requires 8s; grok-imagine-1.5: 480p/720p/1080p — 1080p not with referenceImages; hailuo-3: 720p/1080p; wan3 & wan3-prime: 480p/720p/1080p; gemini-omni-flash: fixed 720p). COST GUARDRAIL: seedance-2 `4k` costs roughly 4x `1080p`. ONLY set `4k` when the user explicitly asks for 4K (or "2160p"/"four K"). Never infer `4k` from vague quality requests like "high quality", "best", "high-res", or "crisp" — use the default or `1080p` for those.

generate_video

Gets one visible Brand Kit from the connected workspace, including its curated image, video, and audio assets grouped by category. Call this when a kit has been selected, when there is one clearly applicable kit, or when the user asks what is inside a kit. Image and video asset URLs can be passed unchanged into the existing generation tools as references. Reading Brand Kits is free and non-destructive.

get_brand_kit

Gets a Runway workflow by id (name, description, updated time). Set `include_graph: true` to also load the saved graph. Optional `version` loads a specific version instead of latest. The graph JSON is for your next tool call — summarize the pipeline for the user; do not dump node IDs or ASCII diagrams unless they ask how it is wired. Do not invent graph JSON. `list_workflows` → `get_workflow` include_graph: true and copy nodes/edges. Read `runway://docs/workflows/authoring` before composing or changing a graph, `runway://docs/workflows/examples` for the API shape, and `runway://docs/workflows/models` before image/video chains.

get_workflow

Lists the connected workspace's visible Brand Kits with their category names and descriptions, without loading assets. Use this for cheap discovery when the user asks for branded work and no Brand Kit is already in context. Call `get_brand_kit` only after a kit is selected or when there is a single clearly applicable kit. An empty list means this workspace has no visible Brand Kits.

list_brand_kits

Lists Runway editor workflows in the authenticated workspace (name, id, updated time). Does not include the graph — use `get_workflow` with `include_graph: true` to inspect. If a similar workflow already exists, reuse it (`get_workflow` / `save_workflow_version` / `run_workflow`) instead of creating a duplicate. Do not invent graph JSON. `list_workflows` → `get_workflow` include_graph: true and copy nodes/edges. Read `runway://docs/workflows/authoring` before composing or changing a graph, `runway://docs/workflows/examples` for the API shape, and `runway://docs/workflows/models` before image/video chains.

list_workflows

Lists every Runway workspace the authenticated user belongs to, with role and a compact disabled-model summary per workspace. The MCP connection is pinned to ONE workspace (chosen at sign-in); use this tool to confirm which workspace you are currently posting to and to compare options before suggesting a switch. By default this omits full disabled-model arrays to keep the response compact; pass includeDisabledModels=true only when you need the exact policy list. To switch workspaces, the user must disconnect and reconnect the Runway MCP from their client (this is intentional — it keeps account context and audit logs aligned to a single workspace per session).

list_workspaces

Localize an existing ad image for another market: the on-screen copy is translated into the target language and the ad is regenerated with the same layout, typography feel, product, and visual creative. Use this instead of re-generating an ad from scratch when the user wants the SAME creative in another language. CREDIT MODE: Runway account availability is checked against the currently connected workspace. WHEN TO USE THIS: - "localize this ad for Japan", "translate this banner into Spanish", "give me German/French/Korean versions of this creative", "adapt this ad for the LATAM market". - One call = one language. For several markets, call this once per language (each call is a separate task and a separate charge). - NOT for translating a *document* or plain text (no image involved), NOT for editing anything other than the ad's language (use `generate_image` with referenceImages), NOT for upscaling (`upscale_image`). IMAGES ONLY: there is no video ad localization. If the user asks to localize a video ad, say that isn't supported yet rather than trying `edit_video` — Aleph rewrites frames and will not reliably retype on-screen copy. SOURCE AD REQUIREMENTS: the image must contain readable on-screen text. If it has none, the task fails with "No translatable text found in the reference image" — when that happens, tell the user the ad has no detectable copy to translate instead of retrying. High-resolution sources with clear typography localize far more reliably than heavily stylized layouts. COST: flat 21 credits per call. OUTPUT: a single localized image. HANDLING USER ATTACHMENTS: - ChatGPT: pass an uploaded image as `imageFile`; the tool imports it server-side. NEVER pass local paths like /mnt/data/... as `image.url`. - Claude/Cursor/local agents: upload local files first via `init_upload` -> curl -> `complete_upload`, then pass the returned asset URL as `image.url`. Outputs of `generate_image` can be passed directly as `image.url`. USER-FACING REPLIES: Talk about the localized ad and the language, not the model or task machinery. Parameters: - image: `{url}` — the source ad to localize. Must be a Runway-hosted asset URL (from `complete_upload` or `generate_image`) or a public HTTPS image URL. - imageFile: ChatGPT-only uploaded image file param `{download_url, file_id, mime_type, file_name}`. - targetLanguage: Required. ISO-style code or English name. Supported: ar (Arabic), zh (Chinese (Simplified)), zh-Hant (Chinese (Traditional)), nl (Dutch), en (English), fr (French), de (German), el (Greek), hi (Hindi), id (Indonesian), it (Italian), ja (Japanese), ko (Korean), pl (Polish), pt (Portuguese), ru (Russian), es (Spanish), sv (Swedish), th (Thai), tr (Turkish), uk (Ukrainian), vi (Vietnamese). Regional variants collapse to the base language (e.g. `pt-BR` -> Portuguese); `zh` is Simplified and `zh-Hant` is Traditional Chinese.

localize_ad

Shows the connected workspace's remaining Runway credits plus available plans and top-up options as an inline card. Call it when the user asks about their credits, balance, plans, pricing, or how to buy more credits — and after a generation fails with an out-of-credits or low-credits result. After an out-of-credits failure, pass `creditsNeeded` (the approximate credit cost of that generation, from the failed tool's cost notes) to render a shortfall card. After a generation fails because the plan does not allow it, call it with `gatedCapability` set to the phrase from that error to render an upgrade card. Purchases are completed on runway.com; the card only links out. When the inline card renders, keep your reply to one short sentence — the card already shows the numbers and buttons.

show_plans_and_credits

Lists recent uploaded and generated assets for the authenticated workspace. Returns asset IDs, media types, task IDs when available, and reusable asset URLs. Use this to help the user pick recent images or videos as references.

list_recent

Remove the background from an image: SAM3 segmentation isolates the subject you describe and outputs a PNG with a TRANSPARENT background (alpha channel) — the subject is kept, everything else is removed. CREDIT MODE: Runway account availability is checked against the currently connected workspace. WHEN TO USE THIS: - "remove the background", "cut out the person/product", "make the background transparent", "isolate the subject", "give me a sticker/cutout of X". - NOT for replacing the background with new content or other edits (use `generate_image` with referenceImages), NOT for upscaling (`upscale_image`). For videos use `remove_video_background`. SOURCE IMAGE CONSTRAINTS: at most 4096px per side. Cost: 1 credit per image. If the subject can't be found in the image the task fails with "No detections found" — rephrase the subject and retry once, then tell the user what couldn't be found. OUTPUT: a PNG with an alpha channel (true transparency). HANDLING USER ATTACHMENTS: - ChatGPT: pass an uploaded image as `imageFile`; the tool imports it server-side. NEVER pass local paths like /mnt/data/... as `image.url`. - Claude/Cursor/local agents: upload local files first via `init_upload` -> curl -> `complete_upload`, then pass the returned asset URL as `image.url`. Outputs of `generate_image` can be passed directly as `image.url`. USER-FACING REPLIES: Talk about the cutout (what's being isolated and that the background becomes transparent), not the model or task machinery. Do not mention SAM3 unless the user asks. Parameters: - subject: Required. What to KEEP, described in a few words (e.g. "the person", "the sneaker", "the dog on the left"). Max 200 characters. - image: `{url}` — source image. URL must be a Runway-hosted asset URL (from complete_upload or generate_image) or a public HTTPS image URL. - imageFile: ChatGPT-only uploaded image file param `{download_url, file_id, mime_type, file_name}`.

remove_image_background

Remove the background from a video: SAM3 segmentation tracks the subject you describe across every frame and outputs a .webm video with a TRANSPARENT background (alpha channel) — the subject is kept, everything else is removed. Set `invert: true` to do the opposite: cut the subject OUT and keep the background. CREDIT MODE: Runway account availability is checked against the currently connected workspace. WHEN TO USE THIS: - "remove the background", "cut out the person", "green-screen this video", "isolate the product", "make the background transparent". - NOT for replacing the background with new content (use `edit_video` — it can restyle or swap backgrounds in place), NOT for content generation (`generate_video`), NOT for sharpening (`upscale_video`). SOURCE VIDEO CONSTRAINTS: at most 60 seconds (~1800 frames at 30fps). Cost: 1 credit/second of input video. `video.durationSeconds` is REQUIRED — the task is billed by duration and the backend rejects sources without one. OUTPUT: a .webm file with an alpha channel. Warn the user that some players show transparent regions as black/gray; the transparency is real and works in editors and compositing tools. AVAILABILITY: account-gated. If the task is rejected with an "Unknown task type" error, this workspace does not have access yet — tell the user the feature isn't enabled for their account instead of retrying. HANDLING USER ATTACHMENTS: - ChatGPT: pass an uploaded video as `videoFile`; the tool imports it server-side. NEVER pass local paths like /mnt/data/... as `video.url`. Still pass `video.durationSeconds` (from file metadata). - Claude/Cursor/local agents: upload local files first via `init_upload` -> curl -> `complete_upload`, then pass the returned asset URL as `video.url`. USER-FACING REPLIES: Talk about the cutout (what's being isolated and that the background becomes transparent), not the model or task machinery. Do not mention SAM3 unless the user asks. Parameters: - subject: Required. What to KEEP, described in a few words (e.g. "the person", "the red car", "the dog on the left"). Max 200 characters. - video: `{url?, durationSeconds, assetId?}` — source video. URL must be a Runway-hosted /datasets/ asset URL (from complete_upload) or a public HTTPS video URL; omit it only when passing videoFile. durationSeconds is ALWAYS required. - videoFile: ChatGPT-only uploaded video file param `{download_url, file_id, mime_type, file_name}`. Pass the duration alongside it as `video: {durationSeconds}`. - keepSound: Optional, default true. Keep the source audio track in the output. - invert: Optional, default false. Remove the SUBJECT and keep the background instead.

remove_video_background

Runs a saved Runway workflow (latest version) in the authenticated workspace. Consumes credits. Returns immediately with a task id — the inline MCP App viewer polls progress when available. Only call `get_task` to poll when no inline viewer is present or the user asks for status. Optional `node_ids` runs only those workflow nodes (upstream must already have succeeded). Optional `node_outputs` overrides constant/asset values for this run without saving a new version. Use `get_workflow` with `include_graph: true` first if you need node ids. Runway account availability is checked against the currently connected workspace. Do not invent graph JSON. `list_workflows` → `get_workflow` include_graph: true and copy nodes/edges. Read `runway://docs/workflows/examples` for the API shape; `runway://docs/workflows/models` before image/video chains.

run_workflow

Saves a new version of an existing workflow graph. Graph must be API JSON copied from `get_workflow` — not `{ type, config }`. Keep the graph minimal and its `nodeProps.position` layout readable: inputs left, processing center, outputs right, with no overlaps. Optional `expected_version` guards against stale edits (must be within 1 of latest). Response includes an editor URL. Validate first with `validate_workflow_graph`. Do not invent graph JSON. `list_workflows` → `get_workflow` include_graph: true and copy nodes/edges. Read `runway://docs/workflows/authoring` before composing or changing a graph, `runway://docs/workflows/examples` for the API shape, and `runway://docs/workflows/models` before image/video chains.

save_workflow_version

Call this when you (the AI agent) get stuck using Runway tools. Examples of when to call: - A generation failed or produced unexpected results - You couldn't figure out which tool to use - The user said the result wasn't what they wanted - The prompting guide didn't have the info you needed This helps the Runway team improve the tools. Be specific about what went wrong.

feedback

Initialize a file upload to Runway. Returns temporary upload URLs for direct upload. USER-FACING REPLIES: Do not mention provider-specific storage services. Say "Runway upload", "Runway-hosted asset", or "temporary upload URL" instead. USE THIS ONLY WHEN: The user has attached a NEW local file (image or video) that isn't already on Runway. CHATGPT: If ChatGPT supplies the uploaded file as `file`, pass it directly. This tool will download the temporary ChatGPT file server-side, upload it to Runway, complete the upload, and return the final asset URL. Do not ask the user to run curl. CHATGPT: Do NOT call this tool with only `filename`, `fileSize`, and `mimeType`; that only returns curl instructions and cannot upload bytes from ChatGPT. For image-to-video, call `generate_video` with `startImageFile`. For video-to-video, call `generate_video` with `referenceVideoFile`. DO NOT USE THIS FOR: - URLs returned by `generate_image`, `generate_video`, or `generate_multishot_video`. Those URLs are already stable Runway-hosted assets — pass them directly as `startFrame.url` / `referenceImages[].url` / `referenceVideo.url`. Re-uploading wastes a turn and creates a duplicate asset. - Any `https://` URL the user pasted that is publicly fetchable. Pass it through directly. CLAUDE/CURSOR/LOCAL WORKFLOW: 1. Get the file size and MIME type using bash: ```bash wc -c < "/absolute/path/to/your/file" | tr -d ' ' file --mime-type -b "/absolute/path/to/your/file" ``` 2. Call this tool with the filename, fileSize, and mimeType. 3. Use bash/curl to upload the file to the temporary upload URL(s). The curl commands this tool returns include `-D -` so the response headers (including ETag) are written to stdout. 4. Capture the ETag header from each PUT response (it looks like `etag: "abc123..."`; strip the surrounding quotes). 5. Call `complete_upload` with the uploadId AND the parts array — `parts` is required even for single-part uploads. 6. Use the returned assetUrl as startFrame.url / referenceImages[].url / referenceVideo.url in the generation tools. Parameters: - file: ChatGPT-only uploaded file param `{download_url, file_id, mime_type, file_name}`. If present, the server completes the upload and returns an asset URL directly. - filename: Name for the uploaded file (e.g., "video.mp4") - fileSize: Size in bytes (from wc -c) - mimeType: MIME type (from file --mime-type). Supported: image/jpeg, image/png, image/webp, image/gif, video/mp4, video/quicktime, video/webm

init_upload

Upscale an existing image to a higher resolution (2x, 4x, 8x, or 16x) using Runway's AI image upscaler. Use this to sharpen, denoise, and increase the resolution of an image — it does NOT change the content, composition, or style; for those edits use `generate_image` with `referenceImages`. CREDIT MODE: Runway account availability is checked against the currently connected workspace. PAID PLAN REQUIRED: This upscaler is not available on free Runway workspaces. If the connected workspace is on a free plan the task will be rejected with an account-limitation message — tell the user image upscaling isn't available in their current workspace and do not retry. There is no free upscaling alternative; do NOT try to fake an upscale with `generate_image`. WHEN TO USE: - The user wants a low-res / soft / blurry / grainy image made crisper or larger (e.g. "upscale this to 4x", "make this image higher resolution", "enhance / denoise this photo"). - The user has a finished image (their own upload, or an output from `generate_image`) and wants a higher-fidelity version of the exact same image. WHEN NOT TO USE: - To restyle, edit, remove/replace backgrounds or objects, or otherwise change the content → use `generate_image` with `referenceImages[0]`. - To generate a brand-new image → use `generate_image`. - To upscale a video → use `upscale_video`. INPUT: - Provide the source image as `image.url`: a Runway-hosted URL (from `complete_upload` or a `generate_image` output) or a public HTTPS image URL. The tool imports it into Runway automatically. - Pass `image.width`/`image.height` when known so scale-factor limits can be checked before submitting. The input must be at least 300x300px, and the output (width * scaleFactor x height * scaleFactor) must stay under ~25 megapixels — e.g. a 4000x3000 image only supports 1 step of 2x. FLAVOR / SCALE RULES: - flavor `sublime` (DEFAULT — artistic/illustrated images, smooth gradients, vibrant colors): scaleFactor 2, 4, 8, or 16. - flavor `photo` (photographic images, natural colors, realistic detail): scaleFactor 2 only. - flavor `photo_denoiser` (noise reduction for low-light/grainy photos): scaleFactor 2 only. HANDLING USER ATTACHMENTS: - Claude/Cursor/local agents: if the user attached a local image, run `init_upload` -> curl -> `complete_upload` first, then pass the returned URL as `image.url`. - ChatGPT: if it supplies an uploaded image as `imageFile`, pass it directly. NEVER pass local paths such as `/mnt/data/file.png` as `image.url`; use `imageFile` instead. USER-FACING REPLIES: Talk about the result ("upscaling your image 4x"), not the underlying engine. Do not mention the model/provider unless the user explicitly asks. Parameters: - image: `{ url, width?, height? }` — the source image to upscale. URL must be public HTTPS or Runway-hosted. Do NOT use ChatGPT local paths like `/mnt/data/...`; use `imageFile` for ChatGPT uploads. - imageFile: ChatGPT-only uploaded image file param `{download_url, file_id, mime_type, file_name}`. - scaleFactor: 2, 4, 8, or 16. Default: 2. Factors above 2 require flavor `sublime`. - flavor: `sublime` | `photo` | `photo_denoiser`. Default: `sublime`. Pick `photo` for photographs, `photo_denoiser` for grainy/low-light photos. - sharpen: 0-100. Default: 10. Higher = crisper edges. - smartGrain: 0-100. Default: 10. Adds natural film-like grain to avoid a plasticky look. - ultraDetail: 0-100. Default: 30. Higher = more invented micro-detail (can drift from the source).

upscale_image

Upscale an existing video to a higher resolution (up to 4K) using Runway's AI video upscaler. Use this to sharpen, clean up, and increase the resolution of footage — it does NOT change the content, motion, style, or duration; for those edits use `generate_video` (video-to-video). CREDIT MODE: Runway account availability is checked against the currently connected workspace. WHEN TO USE: - The user wants a low-res / soft / blurry video made crisper or larger (e.g. "upscale this to 4K", "make this video higher resolution", "enhance the quality of this clip"). - The user has a finished clip (their own upload, or an output from another tool that they re-uploaded) and wants a higher-fidelity version of the exact same video. WHEN NOT TO USE: - To restyle, edit, remove/replace backgrounds, remove objects, or otherwise change the content → use `generate_video` with `referenceVideo`. - To generate a brand-new video → use `generate_video`. INPUT: - Provide the source video as `video.url`. This must be a Runway-hosted `/datasets/<UUID>.<ext>` URL returned by `complete_upload`, OR a public HTTPS video URL (the tool imports it into Runway automatically). - Pass `video.durationSeconds` when known. Videos must be under 40 seconds. HANDLING USER ATTACHMENTS: - Claude/Cursor/local agents: if the user attached a local video, run `init_upload` -> curl -> `complete_upload` first to get a Runway-hosted asset URL, then pass it as `video.url`. - ChatGPT: if it supplies an uploaded video as `videoFile`, pass it directly. This tool will download the temporary ChatGPT file server-side, upload it to Runway, and use it as the source. NEVER pass local paths such as `/mnt/data/file.mp4` as `video.url`; use `videoFile` instead. REUSING ASSETS: A URL from `complete_upload` is a stable, hosted asset URL — pass it directly as `video.url`, never re-upload. Note: outputs of `generate_video` are NOT yet usable as `video.url` here (re-upload via `init_upload` if needed). USER-FACING REPLIES: Talk about the result ("upscaling your video to 4K"), not the underlying engine. Do not mention the model/provider unless the user explicitly asks. Parameters: - video: `{ url, durationSeconds? }` — the source video to upscale. URL must be public HTTPS or a Runway-hosted `/datasets` URL. Do NOT use ChatGPT local paths like `/mnt/data/...`; use `videoFile` for ChatGPT uploads. - videoFile: ChatGPT-only uploaded video file param `{download_url, file_id, mime_type, file_name}`. Use this instead of `video` only when ChatGPT provides a user-uploaded file.

upscale_video

Validates a workflow graph without saving. Use before `create_workflow` or `save_workflow_version`. Graph must be API JSON (UUID node ids, `nodeType`, `nodeOutputs`) — copy from `get_workflow` include_graph, do not invent `{ type, config }`. Set `estimate_cost: true` to also fetch a credit estimate for the graph. Do not invent graph JSON. `list_workflows` → `get_workflow` include_graph: true and copy nodes/edges. Read `runway://docs/workflows/authoring` before composing or changing a graph, `runway://docs/workflows/examples` for the API shape, and `runway://docs/workflows/models` before image/video chains.

validate_workflow_graph