Creating corpus can be tricky in Google where we can get error messages about corpus type not being avaiable in certain regions.
Here is a quick example where we can create this corpus using console.
Click on "Create corpus"
And then we need to provide some details :-
Please check to see corpus is supported in your region
Next, click on upload to upload your document. Supported documents are pdf, html, jsonl, json, markdwn, textfile, docs, pptx, google docs. No CSV :). Size are typically 10 to 50 MB.
This is different from vertex search which does support csv.
There are 3 embedding model here but we can only choose 2 (text-embedding-005 and text-multilingual-embedding-002) that are recommended for use with RAG.
Something to note: As you can see we are running in "serverless mode" and that does not support Vector Search (At least at the moment).
Ensure you have selected "Managed Agent Retrieval".
And click "
create corpus".
Here is the script version to re-create the same instance. You can create multiple corpus with the same name (it will have a unique instance id that is made up of project id - just numbers version as shown here)
projects/numeric-project-id/locations/us-central1/ragCorpora/2228710069501952000
We are using Vertex AI ADK to create the corpus. To use the approach dictated here.
import agentplatform
from agentplatform import types
from google import genai
from google.genai import types as genai_types
# Create a RAG Corpus, Import Files, and Generate a response
PROJECT_ID = "project-your-id"
MODEL_ID = "gemini-3.5-flash"
display_name = "test_corpus"
gcs_path = "gs://your-bucket/cars.csv"
google_drive_path ="gs://your-bucket"
# Initialize Agent Platform client once per session
client = agentplatform.Client(project=PROJECT_ID, location="us-central1")
# Configure embedding model, for example "text-embedding-005".
embedding_model_config = types.RagEmbeddingModelConfig(
vertex_prediction_endpoint=types.RagEmbeddingModelConfigVertexPredictionEndpoint(
endpoint="publishers/google/models/text-embedding-005"
),
)
# Create RagCorpus
rag_corpus = client.rag.create_corpus(
rag_corpus=types.RagCorpus(
display_name=display_name,
rag_vector_db_config=types.RagVectorDbConfig(
rag_embedding_model_config=embedding_model_config
)
)
)
# Import Files to the RagCorpus
client.rag.import_files(
name=rag_corpus.name,
import_config=types.ImportRagFilesConfig(
gcs_source=genai_types.GcsSource(uris=[gcs_path]),
rag_file_transformation_config=types.RagFileTransformationConfig(
rag_file_chunking_config=types.RagFileChunkingConfig(
chunk_size=512,
chunk_overlap=100,
)
), # optional
max_embedding_requests_per_min=1000, # optional
)
)
# Direct context retrieval
rag_retrieval_config = genai_types.RagRetrievalConfig(
top_k=3, # Optional
filter=genai_types.RagRetrievalConfigFilter(
vector_distance_threshold=0.5
), # Optional
)
response = client.rag.retrieve_contexts(
vertex_rag_store=genai_types.VertexRagStore(
rag_resources=[
genai_types.VertexRagStoreRagResource(
rag_corpus=rag_corpus.name,
)
],
),
query=types.RagQuery(
text="What is RAG and why it is helpful?",
rag_retrieval_config=rag_retrieval_config,
)
)
print(response)
print("Rag Corpus Name:", rag_corpus.name)
# Enhance generation
# Create a RAG retrieval tool
rag_retrieval_tool = genai_types.Tool(
retrieval=genai_types.Retrieval(
vertex_rag_store=genai_types.VertexRagStore(
rag_resources=[
genai_types.VertexRagStoreRagResource(
rag_corpus=rag_corpus.name,
# Optional: supply IDs from `rag.list_files()`.
# rag_file_ids=["rag-file-1", "rag-file-2", ...],
)
],
rag_retrieval_config=rag_retrieval_config,
),
)
)
And if we are going to test it (we are not setting up an agent to use the rag corpus) for that you need to use a different library in ADK.
The client to use this corpus.
# Direct context retrieval
rag_retrieval_config = genai_types.RagRetrievalConfig(
top_k=3, # Optional
filter=genai_types.RagRetrievalConfigFilter(
vector_distance_threshold=0.5
), # Optional
)
response = client.rag.retrieve_contexts(
vertex_rag_store=genai_types.VertexRagStore(
rag_resources=[
genai_types.VertexRagStoreRagResource(
rag_corpus="projects/numeric-project-id/locations/us-central1/ragCorpora/2228710069501952000",
)
],
),
query=types.RagQuery(
text="What is RAG and why it is helpful?",
rag_retrieval_config=rag_retrieval_config,
)
)
print(response)
# Enhance generation
# Create a RAG retrieval tool
rag_retrieval_tool = genai_types.Tool(
retrieval=genai_types.Retrieval(
vertex_rag_store=genai_types.VertexRagStore(
rag_resources=[
genai_types.VertexRagStoreRagResource(
rag_corpus="projects/numeric-project-id/locations/us-central1/ragCorpora/2228710069501952000",
# Optional: supply IDs from `rag.list_files()`.
# rag_file_ids=["rag-file-1", "rag-file-2", ...],
)
],
rag_retrieval_config=rag_retrieval_config,
),
)
)
# Call generate_content with the tool using the GenAI SDK
# Create a GenAI SDK client
genai_client = genai.Client(enterprise=True, project="project-id", location="us")
response = genai_client.models.generate_content(
model=MODEL_ID,
contents="What is RAG and why it is helpful?",
config=genai_types.GenerateContentConfig(
tools=[rag_retrieval_tool]
)
)
print(response.text)
Comments