Aws

80.1 million requests per second on DynamoDB

Amazon Live and IVS on Prime Day Throughout Prime Day 2020, Amazon customers were able to shop from livestreams through Amazon Live. Shoppers were also able to use live chat to interact with influencers and hosts in real time. They were able to ask questions, share their experiences, and get a better feel for products of interest to them.

Samsung Migrates 1.1 Billion Users across Three Continents from Oracle to Amazon Aurora

The industry estimates that Samsung Electronics paid more than $483 million to AWS in 2019

Facebook AI, AWS partner to release new PyTorch libraries

Facebook AI and AWS have partnered to release new libraries that target high-performance PyTorch model deployment and large scale model training. As part of the broader PyTorch community, Facebook AI and AWS engineers have partnered to develop new libraries targeted at large-scale elastic and fault-tolerant model training and high-performance PyTorch model deployment. These libraries enable the community to efficiently productionize AI models at scale and push the state of the art on model exploration as model architectures continue to increase in size and complexity.

How Amazon is solving big-data challenges with data lakes

Back when Jeff Bezos filled orders in his garage and drove packages to the post office himself, crunching the numbers on costs, tracking inventory, and forecasting future demand was relatively simple. Fast-forward 25 years, Amazon’s retail business has more than 175 fulfillment centers (FC) worldwide with over 250,000 full-time associates shipping millions of items per day. Amazon’s worldwide financial operations team has the incredible task of tracking all of that data (think petabytes).

Database Migration To Amazon Aurora

In this blog post we’ll show you how we migrated a critical Postgres database with 18Tb of data from Amazon RDS (Relational Database Service) to Amazon Aurora, with minimal downtime. To do so, we’ll discuss our experience at Codacy.

Cloud-Powered, Next-Generation Banking

Traditional banks make extensive use of labor-intensive, human-centric control structures such as Production Support groups, Security Response teams, and Contingency Planning organizations. These control structures were deemed necessary in order to segment responsibilities and to maintain a security posture that is risk averse. Unfortunately, this traditional model tends to keep the subject matter experts in these organizations at a distance from the development teams, reducing efficiency and getting in the way of innovation.

Amazon Releases Container Monitoring for Amazon ECS, EKS, and Kubernetes via CloudWatch

AWS power outage with data loss

On August 31st, 2019, an Amazon AWS US-EAST-1 datacenter in North Virginia experienced a power failure at 4:33 AM, which led to the datacenter’s backup generators to kick on. Unfortunately, these generators started failing at approximately 6:00 AM , which led to 7.5% of the EC2 instances and EBS volumes becoming unavailable. ‘1:30 PM PDT At 4:33 AM PDT one of ten data centers in one of the six Availability Zones in the US-EAST-1 Region saw a failure of utility power.

Toward a bastion-less world

Using a bastion or jump server has been a common way to allow access to secure infrastructure in your virtual private cloud (VPC) and is integrated into several Quick Starts. Amazon Web Services (AWS) has recently released two new features that allow us to connect securely to private infrastructure without the need for a bastion host. This greatly improves your security and audit posture by centralizing access control and reducing inbound access.

A Technical Analysis of the Capital One Hack

The recent disclosure of yet another cloud security misconfiguration leading to the loss of sensitive personal information made the headlines this past week. This particular incident came with a bit more information from the indictment of the accused party, allowing us to piece together the revealed data and take an educated guess as to what may have transpired leading up to the loss of over 100 million credit card applications and 100 thousand social security numbers. At the root of the hack lies a common refrain: the misconfiguration of cloud infrastructure resources allowed an unauthorized user to elevate her privileges and compromise sensitive documents.

The Technical Side of the Capital One AWS Security Breach

Key Conjurer: Our Policy of Least Privilege

Hi, my name is Reza Nikoopour and I’m a security engineer on the Security team at Riot. My team is responsible for securing Riot infrastructure wherever we’re deployed – whether that means internal or external data centers or clouds. We provide cloud security guidance to the rest of Riot, and we’re responsible for Key Conjurer, our open source AWS API programmatic access solution.

VPC Traffic Mirroring – Capture & Inspect Network Traffic

Running a complex network is not an easy job. In addition to simply keeping it up and running, you need to keep an ever-watchful eye out for unusual traffic patterns or content that could signify a network intrusion, a compromised instance, or some other anomaly. VPC Traffic Mirroring Today we are launching VPC Traffic Mirroring.

AWS Control Tower – Set up & Govern a Multi-Account AWS Environment

AWS Security Hub Now Generally Available

I’m a developer, or at least that’s what I tell myself while coming to terms with being a manager. I’m definitely not an infosec expert. I’ve been paged more than once in my career because something I wrote or configured caused a security concern.

Architecting for PCI DSS Segmentation and Scoping on AWS

AWS has published a whitepaper, Architecting for PCI DSS Scoping and Segmentation on AWS, to provide guidance on how to properly define the scope of your Payment Card Industry (PCI) Data Security Standard (DSS) workloads running on the AWS Cloud. The whitepaper looks at how to define segmentation boundaries between your in-scope and out-of-scope resources using cloud native AWS services. The whitepaper is intended for engineers and solution builders, but it also serves as a guide for Qualified Security Assessors (QSAs) and internal security assessors (ISAs) to better understand the different segmentation controls available within AWS products and services, along with associated scoping considerations.

Simplify DNS management in a multi-account environment with Route 53 Resolver

AWS and the CLOUD Act

While news of Brexit dominates headlines in the United Kingdom, another important event took place recently in London. U.S. Deputy Assistant Attorney General Richard W. Downing addressed the myths and realities of the Clarifying Lawful Overseas Use of Data Act (“CLOUD Act”), in a speech at the Academy of European Law Conference. Following the speech, the U.S. Department of Justice (DOJ) published a whitepaper and FAQ clarifying the purpose and scope of the CLOUD Act and addressing many of the misunderstandings of this law.

A Detailed Overview of AWS API Gateway

Look inside the black box of AWS API Gateway to understand authorization, method requests and responses, integration requests and responses, VTL templates, and more. AWS API Gateway is an awesome service to use as an HTTP frontend. You can use it for building serverless applications, for integrating with legacy applications, or for proxying HTTP requests directly to other AWS services.

Disaster Tolerance Patterns Using AWS Serverless Services

In my previous post (Disaster Recovery for Cloud Solutions is Obsolete) I asserted that you should design your cloud architectures for Disaster Tolerance from the start (even if it is counter intuitive to do so by lean principles). I also argued that you should do this because it’s easy if you do it now, and it will help your business even if there is never a disaster.

6 new ways to reduce your AWS bill with little effort

AWS App Mesh

AWS recently released a new service App Mesh during the 2019 summit which has generated a lot of interest from developers world-wide. This service is a great example of how Amazon is highly customer-focused in delivery of products/features to the market. Besides that, there is no additional charge for using the service!:-)

AWS App Mesh—Service Mesh for Microservices Running on AWS

The idea of a “service mesh” has become increasingly popular over the last couple of years and the number of alternatives available has risen. There are multiple service mesh open-source projects: Istio, Linkerd, Envoy and Conduit which can be deployed on any Kubernetes environment. The AWS App Mesh can be used with microservices running on Amazon Elastic Container Service (Amazon ECS), Amazon Elastic Container Service for Kubernetes (Amazon EKS), and Kubernetes running on Amazon EC2.

AWS App Mesh is now generally available

AWS App Mesh is now generally available and supported for production use. App Mesh is a service mesh that provides application level networking to make it easy for your services to communicate with each other across multiple types of compute infrastructure. App Mesh standardizes how your services communicate, giving you end-to-end visibility and ensuring high-availability for your applications.

Amazon S3 Path Deprecation Plan

Last week we made a fairly quiet (too quiet, in fact) announcement of our plan to slowly and carefully deprecate the path-based access model that is used to specify the address of an object in an S3 bucket. I spent some time talking to the S3 team in order to get a better understanding of the plan. We launched S3 in early 2006.

Packets-per-second limits in EC2

While evaluating the performance of a server application, we eventually (and hopefully) run up against the most fundamental constraining factor: the network. Cloud providers tend to offer somewhat handwavy guidance on networking constraints, especially when compared to the exhaustive literature explaining the quotas for RAM, CPU, and I/O. While working on an unrelated stress test in EC2, we were surprised by some results that led us down the path of investigating EC2 network capacity claims, resulting in this writeup. EC2 documentation describes network performance in terms of maximum available bandwidth.

When AWS Autoscale Doesn’t

The premise behind autoscaling in AWS is simple: you can maximize your ability to handle load spikes and minimize costs if you automatically scale your application out based on metrics like CPU or memory utilization. If you need 100 Docker containers to support your load during the day but only 10 when load is lower at night, running 100 containers at all times means that you’re using 900% more capacity than you need every night. With a constant container count, you’re either spending more money than you need to most of the time or your service will likely fall over during a load spike.

New AWS services launch with HIPAA, PCI, ISO, and SOC

The Guardian’s Migration from MongoDB to PostgreSQL on Amazon RDS

New whitepaper: Achieving Operational Resilience in the Financial Sector and Beyond

Western Digital HDD Simulation at Cloud Scale – 2.5 Million HPC Tasks, 40K EC2 Spot Instances

AWS Outperforms GCP in the 2018 Cloud Report

Customers rely on us to help navigate the complexities of the increasingly competitive cloud wars. This inspired the 2018 Cloud Computing Report, where we benchmark performance, latency, CPU, network, I/O, and cost of AWS and GCP. Note: As of December 20, 2018, we have updated two sections in this report: Network Throughput and I/O Experiment.

Moving from Kube2Iam to Kiam

At Ibotta, we chose kube2iam to assign AWS IAM Roles to containers running in our Kubernetes cluster. Lately, we’ve run into some issues with it—specifically when running a job that scores all of our service repos. This spins up a number of pods in parallel and has often failed to correctly access roles.

Amazon Introduces Intelligent-Tiering for S3 Storage to Automatically Optimize Costs

Crossplane lowers the barrier to a multi-cloud future

Cloud computing has become the dominant IT paradigm and multi-cloud looks poised to be the primary approach, with 81 percent of enterprises already adopting a multi-cloud strategy. A multi-cloud strategy prevents vendor lock-in, which is increasingly important as three major providers (AWS, GCP, and Azure) dominate the market. Despite the many benefits of a multi-cloud strategy, deploying across multiple clouds is still incredibly complex.

Amazon is coming to company data centers, and that should scare Cisco, HPE and Dell

Predictive Scaling for EC2, Powered by Machine Learning

When I look back on the history of AWS and think about the launches that truly signify the fundamentally dynamic, on-demand nature of the cloud, two stand out in my memory: the launch of Amazon EC2 in 2006 and the concurrent launch of CloudWatch Metrics, Auto Scaling, and Elastic Load Balancing in 2009. The first launch provided access to compute power; the second made it possible to use that access to rapidly respond to changes in demand. We have added a multitude of features to all of these services since then, but as far as I am concerned they are still central and fundamental!

Deploy Gluon models to AWS DeepLens using a simple Python API

Comparing AWS Lambda performance of Node.js, Python, Java, C# and Go

Spooky action at a distance, how an AWS outage ate our load balancer