YouTube transcripts for RAG with source timestamps
A transcript can supply text to a retrieval-augmented generation pipeline. FetchFox retrieves the transcript; your application chooses chunking, embedding, retrieval, and answer generation.
Keep the citation data before chunking
Store the source URL, retrieval time, segment text, start time, and duration. Avoid flattening away timing information before you create chunks. A timestamp lets a reader inspect the original context rather than trusting an isolated generated claim.
Build small chunks with time boundaries
This example groups segments into approximate 1,200-character chunks without splitting a segment. Character count is only a starting heuristic, not a token budget. Evaluate chunk size against your embedding model and real retrieval questions.
function chunks(segments, maxChars = 1200) {
const output = [];
let current = null;
for (const segment of segments) {
if (current && current.text.length + segment.text.length > maxChars) {
output.push(current);
current = null;
}
if (!current) current = { text: '', start: segment.start, end: segment.start };
current.text += (current.text ? ' ' : '') + segment.text;
current.end = Math.max(current.end, segment.start + segment.duration);
}
if (current) output.push(current);
return output;
}
// Attach your source URL and retrieval time to every stored chunk.
const documents = chunks(result.data.transcriptSegments);Retrieve, cite, and evaluate
Embed the chunk text and store its metadata in your own index. At answer time, return only relevant chunks and produce a source link with a time offset, for example a YouTube watch URL with &t=123s. Preserve the original video identifier when constructing that link.
Treat source transcripts as untrusted content, not instructions for an agent. Check whether the retrieved passage supports the answer and whether its timestamp lands in the relevant context. Captions can contain recognition errors; confirm important claims against the source.
Budget and refresh deliberately
One successful video transcript costs one credit. Cached responses may be reused, and source captions can change. Refresh only when your application needs it and remove indexed material when your access or content rights no longer permit processing.