Implement computer vision solutions
Image and Video Generation and Editing
ImportantGenerate images and videos from text and reference media with Foundry image and video models, edit images with masks and prompts, revise generated videos, and choose the size, quality, format, and duration controls each model supports.
Aligned to the AI-103 study guide with skills measured as of April 16, 2026; guide and product behavior verified October 12, 2026.
Why this matters
Marketing, product, and training teams increasingly ask developers for generated visuals. Choosing a retired or limited-access model, sending an edit without a correct mask, or expecting a synchronous result from an asynchronous video job are common failure points that AI-103 scenarios probe.
Must Know
- Image generation in Foundry uses the gpt-image model family through the image generation API or the Responses API. The dall-e-3 model was retired on March 4, 2026, and its deployments no longer work.
- As of October 2026, gpt-image-2 and the gpt-image-2.5 models are generally available, while gpt-image-1, gpt-image-1-mini, and gpt-image-1.5 are limited-access preview models that require registration.
- Generation controls include size, quality (for example low, medium, or high), n for one to ten images per request, output_format (PNG or JPEG), output_compression for JPEG, and background set to transparent for PNG output.
- The image edit API accepts one or more input images plus a prompt. For inpainting, supply a PNG mask with the same dimensions as the image; fully transparent pixels mark the area to change.
- input_fidelity set to high keeps faces, style, and unedited areas closer to the input image during edits; it is not supported by gpt-image-1-mini.
- Video generation with Sora 2 (preview) is asynchronous: you create a video job, poll its status until it completes, and then download the MP4. It supports text-to-video and image-to-video, where an input reference image anchors the first frame.
- To revise a generated video, remix it by passing the completed video's ID with a new prompt. The remix keeps the original structure and applies the requested change, so make one narrow change per remix.
- Video controls include size (for example 1280x720 landscape or 720x1280 portrait) and duration in seconds, and video output can include audio.
- Generated images include Content Credentials, a signed C2PA manifest that identifies the image as AI-generated.
Compare and Distinguish
- Generation versus editing: create a new image from a prompt versus change an existing image with a prompt and optional mask.
- Mask-based inpainting versus prompt-only editing: change only the transparent region versus let the model revise the whole image as instructed.
- Text-to-video versus image-to-video versus remix: prompt only versus a reference image as the first frame versus revising a video you already generated.
- Quality setting versus size: rendering effort and latency versus pixel dimensions and aspect ratio.
Scenario examples
- Scenario: A furniture site must replace only the sofa in a room photo while leaving the walls and floor untouched. Think: the image edit API with a PNG mask whose transparent area covers the sofa.
- Scenario: A brand team needs a product shot with a transparent background for web compositing. Think: background set to transparent with PNG output.
- Scenario: A generated training clip is good except the presenter's jacket color. Think: remix the completed video with a prompt that changes only the jacket color.
Exam traps
- DALL-E 3 is retired; options that depend on it are wrong for new solutions.
- A mask must match the input image's dimensions, and it is the transparent region, not the opaque region, that gets edited.
- Video generation does not return the file in the create call; you must poll the job and then download the result.
- Transparent backgrounds require PNG output; JPEG cannot carry transparency.
- Sora 2 rejects photorealistic real people and input images that contain human faces.
Key takeaways
- Use current gpt-image models for images and Sora 2 (preview) jobs for video.
- Edit precisely with masks and input fidelity, and revise videos with targeted remixes.
- Pick size, quality, format, and duration to meet the brief and the latency budget.
How it works
- The edit endpoint receives multipart form data with the image, optional mask, and prompt, and returns base64-encoded image data.
- A video job moves through queued and in-progress states to completed or failed, and the completed job exposes downloadable content.
Objects and administrative surfaces
- Foundry portal Images and Video playgrounds with View code.
- Image generation and image edit endpoints, and the Responses API image generation tool.
- Video job endpoints to create, get status, download, list, and delete videos.
- Content Credentials verification tools for C2PA manifests.
When to use it
- Use lower quality settings for fast drafts and higher quality for final assets.
- Use image-to-video when the first frame must match an approved still image.
Security and governance implications
- Image prompts and outputs pass through guardrails, and blocked requests return a content policy error.
- Keep Content Credentials intact when publishing so viewers can verify AI origin.
Troubleshooting signals
- If an edit changes the wrong area, check that the mask's transparent pixels align with the target region and the dimensions match.
- If image calls time out, request a smaller size, lower quality, or streaming of partial images.
More detail
- Generate images from text prompts and reference images.
- Generate videos from text prompts and reference images.
- Configure inpainting, mask-based, and prompt-driven image edits.
- Edit generated videos and select generation and editing controls.
Ready for the quiz?
- Which pixels in a mask image are edited during inpainting?
- What three calls does a video generation workflow need after submission?
- When would you raise input_fidelity for an edit?
- How do you change one detail in a generated video without regenerating it from scratch?
Related objectives
- D3.1.S1 — Implement a solution that generates images from text prompts and reference media
- D3.1.S2 — Implement a solution that generates videos from text prompts and reference media
- D3.1.S3 — Configure image-editing workflows, including inpainting, mask‑based edits, and prompt‑driven modifications
- D3.1.S4 — Implement workflows to edit generated videos
- D3.1.S5 — Select and apply appropriate generation and editing controls provided by the platform