AWS
Money
Free Tier Usage
Regions and Availability Zones:
- Region: physical location. Used to bring resources closer to the consumer
Can replicate between regions for additional $, better availability.
- Availability Zone: Within a region, for availability. Replication done in three isolated availability zones within a region. See S3 video.
- Global Infrastructure
- Explained in S3 Video
Security, Identity & Compliance:
- Setup:
Create root account
Create user under root account. Do everything with this user
or other users you create
Restrict access by users (one security alternative)
Create Access Keys for the user, so you can make REST or HTTP calls
- IAM Console and Video, used to manage users and their permissions/roles.
- Can attach Permissions to users or to roles attached to the user when running a particular resource.
- Example: Managing Access Permissions to Your Amazon S3 Resources
Python API
API are at two levels, both at the application level and at the orchestration level.
That is, there are api for doing pretty much what you can do on the AWS consoles.
Example:
DynamoDB application level API: to create tables, insert, query, update items in a table.
DynamoDB orchestration level: manage table properties, replication, availability
Python API Setup
- pip3 install boto3 --user
- Configure credenitals with ~/.aws/credentials
[default]
aws_access_key_id = YOUR_ACCESS_KEY
aws_secret_access_key = YOUR_SECRET_KEY
- Can configure region with ~/.aws/config:
[default]
region=us-east-1
- boto3 API reference
- Reference: AWS Python Sample,
Quick Start
AWS Python Samples
here and here
Storage: s3 (Files) (Simple Storage Service)
- For storing files in directories.
- Structure: Bucket/directory/key (=file name)
- s3 examples
- video
- console
- s3 Glacier (Archive Storage): pay less for 'offline' storage.
Database: RDS (Postgresql, SQL Serve, ...r)
- DBs supported: Aurora, MySQL, PostgreSQL, MariaDB, Oracle, SQL Server
- Value Proposition: They manage, you don't worry, you comply.
- Scale by adding Read replicas (replication = mirroring in db terminology).
- Availability/Healing
- Replicates in alternate availability zones, failover to standby.
- Run replicas in multiple availability zones or regions
- Disaster recovery: Backup services
- Connect: Through standars SQL client for the specific db
- connecting to a postgresql db
- Getting Started
Database: dynamoDB (NoSQL db), like MongoDB
- For storing JSON objects
- Key-Value store, where you specify the key in the schema.
- Everything else is optional, unstructured value.
- Replication: Can replicate across multiple regions.
- Partition:
- Main table key (Hash Key), used to shard data into 3 datacenters
- Local Secondary Index: After using tables HashKey you can use a alternate range key. So within a particular HashKey, that is in a particular shard.
- Global Secondary Index: Hash and Range key different from the tables. Applies to whole table.
- Consistency: DynamoDB replicates data
Choice of consistency levels
- Eventually consistent reads: This is better for read throughput, but you might read stale data.
- Strongly consistent reads: Used when you absolutely need the latest result.
- Examples:
curl -sL https://s3.amazonaws.com/ddb-deep-dive/dynamodb.tar | tar -xv
- dynamoDB main page
- WHAT IS THIS
- difference between s3 and dynamoDB
Database: ElastiCache (Redis or Memcached)
- console
- Note: Seems like redis has won this fight.
Network & Content Delivery: CloudFront (Global Content Delivery Network)
CloudFront distribution. A distribution allows you to distribute content using
a worldwide network of edge locations that provide low latency and high data transfer speeds
Compute: Lambda
Compute: EC2 (Virtual Servers)
Compute: ECS (Docker containers)
- Value Proposition: They manage the hardware as well as container orchestration, and monitoring.
- Basically same ideas we covered in Docker Containers, AWS uses slightly different terminology.
Task=container
Service=Docker service
Stack=?
Cluster=EC2 instances (servers)
- ECS Console
- Tutorial
- Migrating to aw at Dockers
Communication: SQS
- Reliable message queue communication between sometimes unreliable services.
- SQS Product Page
- Example: SQS
Developer Tools: Cloud9 (Cloud IDE)
Compute: Elastic Beanstalk (Web Apps)
References