Research on aligning AI with human values and intent, and reports documenting model failures. We show the existence of a new variety of prompt injection, which can self-propagate akin to a computer worm. No impact was observed outside of the simulated tool calls in training and evaluation; we are sharing this due to the novel nature of the prompt injection, not because of any incident.
Source: alignment.openai.com
Many data scientists claim that around80% of their time is spent on data preprocessing, and for good reasons, as collecting, annotating, and formatting data are crucial tasks in machine learning. This article will help you understand the importance of these tasks, as well as learn methods and tips from other researchers. Below, we will highlight academic papers from reputable universities and research teams on various training data topics.
Read More
In this post, we share how Claude made the open-source models that scientists use to predict and design biomolecules faster and more memory-efficient. Claude, working within Claude Science, optimized more than 30 of these models in just under four weeks, speeding them up roughly 4x on average. It also created a low-memory mode that enables the accurate prediction of biomolecular systems larger than 10,000 tokens (amino acids, nucleotides, and atoms from small molecules and ions) on a single NVIDIA GPU node. We are open-sourcing all of the optimized code and announcing a protein design competition co-sponsored with Adaptyv Bio, backed by up to $1 million in Claude credits and wet lab validation for over 5,000 designs.
Read More
A deliberately simple watermark model shows how much room a text watermark has: the entropy available at each token, how a small bias becomes detectable, and why length matters.
Read More