OPTIMAL SHARD PLACEMENT IN A PETABYTE SCALE ELASTICSEARCH CLUSTER
The number of shards on each node, and tries to balance the number of shards per node evenly across the clusterThe high and low disk watermarks. Elasticsearch considers the available disk space on a node before deciding whether to allocate new shards to that node or to actively relocate shards away from that node. A nodes that has reached the low watermark (i.e 80% disk used) is not allowed receive any more shards.
Read moreEMBRACING FAILURES AND CUTTING INFRASTRUCTURE COSTS: SPOT INSTANCES IN KUBERNETES
Spot Instances are unused servers that are available for less than the regular price. Therefore, you can significantly save on your infrastructure costs. It does come with a price, though. Your cloud provider can take away your spot instance at any time, and give to another client who has requested it at a standard cost. How can you save money, but work around disappearing servers? Learn how you can leverage Kubernetes to self-heal your infrastructure and cut costs with Spot Instances.
Read moreGRAPHQL: A SUCCESS STORY FOR PAYPAL CHECKOUT
At PayPal, we recently introduced GraphQL to our technology stack. At PayPal, GraphQL has been a complete game changer to the way we think about data, fetch data and build applications. This blog post takes a close look at PayPal Checkout and explains our journey from REST to Batch REST to GraphQL and lessons learned along the way. PayPal’s Checkout products spread across many web and mobile apps, supporting millions of users across ~200 countries and has hundreds of experiments running at any time. These apps leverage the same suite of REST APIs to fetch data needed for building UI. About 4 years ago, we went all in on REST.
Read moreCROSS SHARD TRANSACTIONS AT 10 MILLION REQUESTS PER SECOND
Dropbox stores petabytes of metadata to support user-facing features and to power our production infrastructure. The primary system we use to store this metadata is named Edgestore and is described in a previous blog post, (Re)Introducing Edgestore. In simple terms, Edgestore is a service and abstraction over thousands of MySQL nodes that provides users with strongly consistent, transactional reads and writes at low latency.
Read moreLEARNING CONCEPTS WITH ENERGY FUNCTIONS
We’ve developed an energy-based model that can quickly learn to identify and generate instances of concepts, such as near, above, between, closest, and furthest, expressed as sets of 2d points. Our model learns these concepts after only five demonstrations. We also show cross-domain transfer: we use concepts learned in a 2d particle environment to solve tasks on a 3-dimensional physics-based robot. Many hallmarks of human intelligence, such as generalizing from limited experience, abstract reasoning and planning, analogical reasoning, creative problem solving, and capacity for language require the ability to consolidate experience into concepts, which act as basic building blocks of understanding and reasoning. Our technique enables agents to learn and extract concepts from tasks, then use these concepts to solve other tasks in various domains.
Read moreMYSTERY PARTICLE SPOTTED? DISCOVERY WOULD REQUIRE PHYSICS SO WEIRD THAT NOBODY HAS EVEN THOUGHT OF IT
There was a huge amount of excitement when the Higgs boson was first spotted back in 2012 – a discovery that bagged the Nobel Prize for Physics in 2013. The particle completed the so-called standard model, our current best theory of understanding nature at the level of particles. Now scientists at the Large Hadron Collider (LHC) at Cern think they may have seen another particle, detected as a peak at a certain energy in the data, although the finding is yet to be confirmed.
Read more20 BEST YOUTUBE CHANNELS FOR AI AND MACHINE LEARNING
What are the most interesting and informative YouTube channels about artificial intelligence (AI) and machine learning? Subscribe to these 20 high-quality channels today to stay up to date with the latest AI and machine learning breakthroughs. Siraj Raval: The School of AI is a growing learning community that aims to offer a free, world-class AI education to anyone. Arxiv Insights: Xander Steenbrugge is a machine learning researcher at ML6. His channel summarizes the key points about machine learning, reinforcement learning, and AI in general from a technical perspective, while making them accessible for a bigger audience.
Read moreWHY CHINESE ARTIFICIAL INTELLIGENCE WILL RUN THE WORLD
With Chinese tech giants Baidu, Alibaba, and Tencent focused on developing sophisticated AI-driven systems in the coming decade, the rest of the world can only watch while China builds the computer systems that will run our world in the decades to come. If you’ve been paying attention in the past year, it seems that all anyone can talk about is the coming artificial intelligence boom on the horizon. Whether it’s the Amazon, Google, or Facebook, everyone seems to be getting in on the AI game as fast as they can.
Read moreEPO ISSUES FIRST GUIDELINES ON AI PATENTS
The European Patent Office (EPO) has issued official guidelines on the patenting of artificial intelligence and machine learning technologies. The guidelines became valid on November 1st, 2018. When determining whether the claimed subject-matter satisfies this condition, the guidelines note that expressions such as “support vector machine,” “reasoning engine” or “neural network” may not qualify, as these are regarded as terms for mathematical methods which do not have a unique technical character of their own.
Read moreA NETFLIX WEB PERFORMANCE CASE STUDY
Netflix is one of the most popular video streaming services. Since launching globally in 2016, the company has found that many new users are not only signing up on mobile devices but are also using less-than-ideal connections to do so. By refining the JavaScript used for Netflix.com’s sign-up process and using prefetching techniques, the developer team was able to provide a better user experience for both mobile and desktop users and offer several improvements.
Read more