AWS bedrock inference profile basic setup and test

Inference profile allow us to define and enforce data residency when processing request. You can setup these profile here by selecting "Infer" -> Inference Profile. 



Under the application tab, click on "Create inference profile".  Next filled up your inference details and click "Create application inference profile". Please note that your model is limited to the region. To add more destination region, you need to use the API.



And to use it, we can goto "Test" -> Playground and then "select model". Then please proceed to select your model and then click on "Inference" where you can choose your inference profiles.


And that's how you can create and use your inference profile. 









Comments

Popular posts from this blog

Windows SSH: Permissions for 'private-key' are too open

NodeJS: Error: spawn EINVAL in window for node version 20.20 and 18.20