EC2

Introduction

  • Elastic Compute Cloud (EC2) is a web service that provides resizable compute capacity.

  • Different types of EC2 instance for multiple usage.

    • D: Density Storage (Fileserver, Data Warehousing, Hadoop)

    • R: Ram

    • M: main choice for general purpose

    • C: Compute

    • G: Graphics

    • I: IOPS (NoSQL DBs, Data Warehousing)

    • F: FPGA (Field Programmable Gate Array, hardware acceleration)

    • T: Cheap general purpose

    • P: Graphics, General purpose of GPU (Machine Learning, Bit coin mining)

    • X: Extreme Memory (SAP HANA, Apache Spark)

  • EC2 instance Options:

    • On demand

      • Pay by the second so far

    • Reserved Instance (RI)

      • Have a discount but for consistent usage in 1 ~ 3 years.

      • Types of RI (can be either regional / zonal):

        • Standard RI (Up to 75% off on demand)

          • Cannot modify Instance Family (Ex. m5.2xlarge, m5 is the Instance Family.)

          • Can be sold

        • Convertible RI (Up to 54% off on demand)

          • Can modify Instance Family

          • Can not be sold

        • Scheduled RI

    • Spot

      • To bid whatever price for the instance capacity. Suitable for flexible start and end times application with very low prices (can be 90% off compared to On-demand).

      • If instance terminated manually, cost of the hour is charged. If instance terminated by AWS, cost of the hour is free.

    • Dedicated Instances:

      • No other customers will share your hardware.

    • Dedicated Hosts

      • Physical EC2 server. Suitable for some applications need instance lock-in.

      • Great for software licenses that operate the core, or CPU socket level.

      • Can define host affinity so that instance reboots are kept on the same host.

Feature

  • Security Group

    • To set up allow rules (not deny rules)

    • Many to many relationship to EC2.

    • Default SG allows all inbound / outbound, Manually created Security Groups allow no traffic by default.

    • SG rule is stateful.

  • Placement Groups

    • A strategy to congrol EC2 instance placement

    • Strtegies:

      • Cluster: instances in one AZ for high latency (10 Gbps).

      • Partition: instances in different logical partition. Good for distributed services like Hadoop, Cassandra, Kafka. Up to 7 partitions per AZ, up to 100s of EC2 instances per partition.

      • Spread: instances in different physical rack and in different AZs. Good for critical applications. Limited to 7 instances per placement group.

    • Can move instance into / out of a placement group without termination (stop the instance, CLI to move, restart instance).

  • Termination protection is off by default, can be anabled.

  • PEM Key scenarios

    • Migrate instance to another region with the same PEM key

      • Copy the AMI of your EC2 machine to new region and start up an instance using the AMI.

        • Choose the "Proceed without a keypair" when launching the instance

        • Or import the public key

      • The PEM key is regional

    • When key is stolen

      • Terminate the instance

      • Launch new instance with another SSH key pair

    • Replace a PEM key

      • log in to the instance and update the public key in ~/.ssh/authorized_keys

      • Users can only select an SSH key pair when they launch a new instance.

  • EC2 included metrics:

    • CPU: CPU Utilization + Credit Usage / Balance

    • Network: Network In / Out

    • Status Check:

      • Instance status: check EC2 VM

      • System status: check the underlying hardware

    • Disk: Read / Write for Ops / Bytes (only for instance store)

    • RAM is NOT included in AWS EC2 metrics (must create a custom metric).

  • Hibernation Limit

    • Enable the hibernation feature on the Configure Instance Details page

    • AMI: Linux, Ubuntu

    • EBS with enough space to save RAM data

    • Not compatible with ASG / ECS

CLI

  • Fetching EC2 meta-data

    • Link into EC2 and list all options of meta-data display

      $ curl http://169.254.169.254/latest/meta-data/

    • Displaying public ip.

      $ curl http://169.254.169.254/latest/meta-data/public-ipv4

  • Configure privilege with programmatic credential of IAM Run below command and give Access Key ID and Secret Access Key. The information is stored in EC2 instance so it's not secured.

      $ aws configure

    PS. A good practice is to use IAM Role to be assigned to EC2 instance (when creation or later on), instead of giving credential away (constraint with EC2 instance level).

Trouble Shooting

  • A HTTP server in EC2 with EIP cannot be accessed from on-premise

    • Security Group should have inbound allow for port 80

    • NACL of the subnet should allow 80 for inboud, ephemeral ports for outbound.

    • The subnet of the instance should be associated with the main route table.

PreviousAuto Scaling GroupNextECS

Last updated