A simple and straight forward post on some of the more popular AWS services. I recently picked it up about 6 months ago so I am definitely not a cloud expert. But I’m a huge believer in sharing is caring, so here’s to those who might find it helpful! :)
Working in security, you probably will hear a lot about open S3 bucket. In plain word, S3 bucket are used for storing data. It’s highly durable, super easy to use, has infinite bandwidth and storage space and one of the few AWS service that require absolute zero capacity management.
It’s a highly durable hash table in the cloud where the key can be any string, and the value is any blob of data from 1b-5tb.
Bucket names are globally unique across all AWS customers… so sometimes, when you’re not careful, you could be uploading data to someone else’s bucket. Or…downloading data from someone else’s bucket… which is pretty concerning. Hence, make sure you attach some type of permissions to restrict read/write usage!
The simple concept is that you get a complete computer in the cloud in a matter of second. You can install and run any software. EC2 offers about 256 different instance types and you can pick and chose based on what they are optimized for such as: CPU, memory, network, storage… Etc.
Regarding the security aspect, there are 2 main things you should pay attention to when setting up your EC2 instance.
Security Group aka an individual firewall
- You can control what traffic can go in and out of your instances.
- For example, allowing inbound port 22 for 0.0.0.0 will allow anyone to SSH into your box (with proper credential of course)
VPC ACL aka the Network Firewall
- You can control what goes in and out of your network
Lambda is a code runner in the cloud. Essentially, it doesn’t have an operating system, file system…etc like your EC2 instance does.
However, the idea is that you can upload some code, and Amazon will run that code for you. It will abstracts away everything except for a function interface where you insert the code that you want to run.
This is a GREAT service in which you can treat it as a plug in for your other AWS service. If an AWS service doesn’t have some type of functionality, you can write a Lambda function to make that functionality. It’s pretty amazing!
DynamoDB is an AWS database that is highly durable. It’s a partitioned B-tree data structure that’s pretty similar to Redis (except that it’s immediately consistent and highly durable), things you put in doesn’t get lost.
As far as load balancer on AWS goes…there’re 3 different types:
- Application Load Balancer (ALB)
- Network Load Balancer (NLB)
- Classic Load Balancer -> these one is kind of a legacy thing.
Essentially, this is just a reverse proxy that sit in the middle of your application and the internet. Every request to your app will get handled by the load balancer first, then the load balancer make another request to your application and forward the response from your application to the caller.
The disadvantage of this is that the proxy approach will add a slight milliseconds delay to each request and may not scale quickly enough to handle a large burst of traffic.
This will route network packets rather than proxying the HTTP requests. It act as a sophisticated network router. When a client connect to the server, the server would see the client as if they’re connected directly.
Unlike ALB, NLB can scale up and down in aggregate and is slightly a little less expensive than ALBs.
Both of these support TLS/HTTPS and integrate pretty well with AWS Certificate Manager!
If you created an EC2 instance, or any type of AWS service by clicking through the AWS UI Console… then CloudFormation let you automate that by defining it in a script.
While it will take some initial investment to write a template, the ability to be able to spin up and down different services will help you to easily manage your product. Typically, you want to let CloudFormation to deal with all AWS thing that’s rarely changing aka…static configurations such as: VPC configuration, Security Groups, Load Balancer, IAM roles, deployment pipelines…etc.
This is a highly durable queue in the cloud. Essentially, you put messages on one end, and a consumer take it out from the other end. It’s easy to use and also have a zero capacity management. There’s no limit on the rate of messages enqueued or consumed and no throttling limit either.
This is a great service to use for dispatching any async work IF you’re ok with messages being potentially out of order or having some duplicates.
If you need strict ordering, and having exact one-to-one delivery (aka no duplicate), then use the FIFO option. The only downside is that FIFO has a throughput limit of 300 msgs/sec.
This is a highly durable linked list in the cloud and another option for async work. Kinesis can have MANY consumers (which SQS only have one). It is a lot cheapper than SQS, records get added to list in a stable order, and consumers always get their records in the same order.
The only down side is that there are a lot more operation burden. You’ll have to figure out the sizing, how many shards you need, you’ll have to monitor those shard utilization, when to add more … etc…
So those are some of the services I hear and use the most. However, since I’m a security person, I also wanted to touch on a few of the AWS security services.
CloudTrail will log everything related to your AWS infrastructure. This include account activity, actions taken, command line tools, etc… You can also use CT to detect any unusual activity on your AWS accounts (things such as log in from a foreign IP address, from TOR server…etc). It provide visibility into users and resource activities. Such as what user created what resource, from what IP address, and when. You can utilize this + CloudWatch to automate workflows.
With CloudWatch, you can define that if X happen, then CW execute workflow Y. According to Amazon, you can create a workflow to add a specific policy to an S3 bucket when CloudTrail logs an API calls that make that bucket public. (Remember, public S3 buckets are bad…don’t do it!)
This is a thread detection service that will continuously monitor your account for any malicious or unauthorized behavior and send you alert.
GuardDuty do this through machine learning anomaly detection with some type of integrated threat intel feed to identify and prioritize potential threats. It analyzes billions of events from CloudTrail, VPC Flow Logs, DNS Logs…etc.
This is an automated assessment service that wil assess your applications for any exposure, vulnerabilities, and deviations from best practices. You can enable additional rules from their own provided rules. Some example include: checking access permission to your instance from public internet, vulnerable software version…etc.
This is a tool that I use as it help with command-line auto-completion.
I use terraform to define my template to automate any deployment pipeline.
- Terraform - https://www.terraform.io/
That’s all I have for this post. As I will be working closely with more AWS services in the future, I will continue to write more about my experiences and any tips/tricks!