Maven Robotics wants to steal your robot deployment deal15m ago Positron Valued at $5 Billion in New Funding as Demand for AI Chips Surges25m ago 7 Steps to Become a Forward Deployed Engineer in 202632m ago Amazon partners with OpenAI to let users of Amazon's DSP run ads in ChatGPT in a pilot program limited to the US; ChatGPT Ads hit $1B in ARR last month48m ago D-Matrix Connects Raptor XPUs to NVIDIA AI Factories via NVLink Fusion1h ago Source: Meta AI researcher Andrew Tulloch, who came over from Thinking Machine Lab as one of the highest paid employees last year, is leaving the company (Reed Albergotti/Semafor)59m ago Mother Horrified When Instagram’s AI Scours Social Media for Invasive Information About Her Young Daughters1h ago Mach Industries, which makes unmanned military vehicles and weapons, raised $600M in an extension to its $300M Series C, doubling its valuation to $3.7B1h ago Salesforce Unveils Six-Capability Trusted AI Harness for Enterprises1h ago US closes $1.9bn loan to restart the Iowa nuclear plant powering Google1h ago OpenAI Gives US Agencies 50% Off Models, Ending $1 Per Year Deal1h ago 47,000 job listings reveal the engineering roles that AI is creating1h ago Anthropic's Alignment Science lead says there is a ">10%" chance AI could kill all humans within the next decade and is worried about recursive self-improvement (Evan Hubinger/@evanhub)1h ago Ukraine says AI is moving beyond drones and into strike missiles for the first time1h ago

Maven Robotics wants to steal your robot deployment deal15m ago Positron Valued at $5 Billion in New Funding as Demand for AI Chips Surges25m ago 7 Steps to Become a Forward Deployed Engineer in 202632m ago Amazon partners with OpenAI to let users of Amazon's DSP run ads in ChatGPT in a pilot program limited to the US; ChatGPT Ads hit $1B in ARR last month48m ago D-Matrix Connects Raptor XPUs to NVIDIA AI Factories via NVLink Fusion1h ago Source: Meta AI researcher Andrew Tulloch, who came over from Thinking Machine Lab as one of the highest paid employees last year, is leaving the company (Reed Albergotti/Semafor)59m ago Mother Horrified When Instagram’s AI Scours Social Media for Invasive Information About Her Young Daughters1h ago Mach Industries, which makes unmanned military vehicles and weapons, raised $600M in an extension to its $300M Series C, doubling its valuation to $3.7B1h ago Salesforce Unveils Six-Capability Trusted AI Harness for Enterprises1h ago US closes $1.9bn loan to restart the Iowa nuclear plant powering Google1h ago OpenAI Gives US Agencies 50% Off Models, Ending $1 Per Year Deal1h ago 47,000 job listings reveal the engineering roles that AI is creating1h ago Anthropic's Alignment Science lead says there is a ">10%" chance AI could kill all humans within the next decade and is worried about recursive self-improvement (Evan Hubinger/@evanhub)1h ago Ukraine says AI is moving beyond drones and into strike missiles for the first time1h ago

AI Research Agents7 min reading time

“Some agents will be pursuing their own objectives”: OpenAI’s chief scientist warns AI could trick and blackmail humans

The New Stack (AI)Published 3 days ago

OpenAI's chief scientist Jakub Pachocki warns that AI systems are becoming too complex to fully control and may soon self-improve recursively, potentially pursuing their own goals and manipulating humans. OpenAI is focusing research on recursive self-improvement and developing automated AI researchers to advance AI capabilities.

More on this story

We have started losing control of AI. It’s time to shut it down | Garrison Lovely
4h· The Guardian
OpenAI's chief scientist says AI labs may need to slow down: 'No one is prepared for the consequences'
4d· 3 sources

#ai-safety #astra #jakub-pachocki #openai

More in AI Research

Anthropic's Alignment Science lead says there is a ">10%" chance AI could kill all humans within the next decade and is worried about recursive self-improvement (Evan Hubinger/@evanhub)

Covered by 9 sources1d ago

AI Research4 min read

Suno trained its v6 AI music models with help from Warner and BMG

Covered by 5 sources1d ago

Anthropic researcher Jacob Coxon says he is quitting the AI industry over fears that tech companies are racing to build systems they won't be able to control (Amrith Ramkumar/Wall Street Journal)

Covered by 11 sources2d ago