AI/ML6 min read

OpenAI API Integration

Integrate the OpenAI API: chat completions, streaming, function calling, embeddings, image generation, and production patterns.

openaigptllmai

The OpenAI API provides access to GPT-4o, o1, and other models for text generation, embeddings, and image generation. Production integration requires handling streaming, tool use, rate limits, costs, and fallback strategies.

Use the official Node.js SDK. Stream responses for better UX on long generations. Use tool/function calling for structured data extraction. Cache expensive calls. Set context limits and cost budgets. Have a fallback model for capacity issues.

Workflow

Make chat completion requestsChat completions are the primary API for text generation with conversational context.
Stream responses for better UXNon-streamed responses feel slow for long generations; streaming shows text as it's generated.
Use function/tool calling for structured dataTool calling makes the model call defined functions with typed arguments - no regex parsing needed.
Generate embeddings for semantic searchEmbeddings convert text to vectors for similarity search, clustering, and RAG retrieval.
Handle rate limits and errorsOpenAI rate limits by tokens-per-minute and requests-per-minute; unhandled 429s crash production.

Output Format

Chat completion with system prompt, streaming for long responses, tool calling for structured extraction, embeddings for search, retry/fallback error handling.

Examples

Streaming: `const stream = await openai.chat.completions.stream({ model: "gpt-4o", messages }); for await (const chunk of stream) { process.stdout.write(chunk.choices[0]?.delta?.content || "") }; const finalCompletion = await stream.finalChatCompletion()`

Tool call: `const response = await openai.chat.completions.create({ model: "gpt-4o", tools: [weatherTool], messages }); if (response.choices[0].finish_reason === "tool_calls") { const call = response.choices[0].message.tool_calls[0]; const result = await executeWeatherAPI(JSON.parse(call.function.arguments)) }`

Retry: `for (let i = 0; i < 3; i++) { try { return await openai.chat.completions.create(params) } catch (e) { if (e instanceof RateLimitError) { await sleep(Math.pow(2, i) * 1000) } else throw e } }`

Edge Cases & Gotchas

Context window: GPT-4o has 128K token context; count tokens with `tiktoken` before sending to avoid context_length_exceeded errors on long documents
Prompt injection in user input: if user content is in the system prompt position, sanitize it; use the user role for untrusted content
Vision (image inputs): pass as URL or base64: `{ role: "user", content: [{ type: "image_url", image_url: { url: "data:image/jpeg;base64,..." } }, { type: "text", text: "What do you see?" }] }`

When to Break These Rules

For latency-sensitive applications where streaming is insufficient, consider self-hosted models (Ollama, vLLM) for local inference with no network round-trip.

Reference Files

OpenAI API: https://platform.openai.com/docs/api-reference
OpenAI Node SDK: https://github.com/openai/openai-node
tiktoken: https://github.com/openai/tiktoken
Cookbook: https://cookbook.openai.com/