Setup LLM model for Amazon AWS
We take data security very seriously. Your code will sit on your premises and go to a model that you control, sitting in your cloud.
Step 1: Get Access to Mixtral 8x7B on AWS Bedrock
AWS Bedrock is a fully managed service providing foundational Model Access. It is in scope for SOC2, HIPAA and ISO and is in compliance with GDPR. For more details on data security, click here.
Visit here. Click
Model Access
. SelectMixtral 8x7B Instruct
and hit “Save Changes”You should see the below after a few seconds.
![](https://framerusercontent.com/images/kHONxF9XvGtaOYQ2hH2w6tSpXv8.png)
Note: Mixtral 8x7B is a completely open source model that is publicly available. This means none of your data is used to train Mixtral 8x7B.
Step 2: Give us access to Mixtral 8x7B on AWS Bedrock
Option #1: via IAM user
You will have to create an IAM user so that p0 can access AWS Bedrock on your behalf.
Click here. Create an user name (
bedrock
). Click Next.Click
Attach Policies Directly
. Search for Bedrock. SelectAmazonBedrockFullAccess
Click
Next
Click
Create User
Change your IAM user in the link here and paste in your browser:
Create
Access Key
—>Local Code
—>Next
—>Create access key
. Keep yourAccess Key
andSecret Access
handy for the product.
Option #1: via IAM user
Option #2: via IAM role
Create a new IAM Policy for Bedrock access
Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam/
In the navigation pane on the left, choose
Policies
.Choose
Create policy
.In the Policy editor section, choose the
JSON
option.Copy and paste the policy mentioned below in the
Policy editor
. In case your region of AWS instance is different, edit it in the policy below underResources
attribute where it saysus-west-2
.
Click
Next
at the bottom-right of the pageAdd the
Policy name
. Make sure to remember this as it will be used later. You can leave the rest of the settings as is.Hit
Create policy
button at the end of the page.
Assign the above policy to an IAM role
Click here to create a new IAM role.
OR, to use an existing IAM role, make sure it has
Amazon EC2
Use case enabled and follow the below steps:Open the details page of the
IAM role
you want to use with your EC2 instance.On the roles detail page, look for add
Add Permissions
in middle-right of the page.In the dropdown, select
Attach Policies
. You will be taken to theAttach Policy to Role
screen.
Here, search for the policy you just created in the above step and select it. Then, click
Add Permission
button at the bottom-right of the screen.
Create an EC2 IAM role with Bedrock policy
Sign in to the AWS Management Console and open the IAM console at https://console.aws.amazon.com/iam
In the navigation pane, select
Roles
, then selectCreate role
.For Select trusted entity, under Trusted entity type, choose
AWS service
.Under Use case, choose
Amazon EC2
, then selectNext
.For Add permissions, select the checkbox for the policy you have created for Bedrock from the policy list, then select
Next
.Enter a name for the role, then select
Create role
. Remember this name because you'll need it when you create your Amazon EC2 instance.