- While jailbreaks and security bypasses are a concern in mainstream AI products, the AI market is gradually filling with AI models that lack the proper protections to prevent abuse.
- Unfortunately, some of them with restrictions that are trivial to bypass are also enjoying increased popularity.
Cybersecurity intelligence firm KELA has recently published the details about the lax security measures for DeepSeek R1 and Alibaba's Qwen 2.5, which are vulnerable to prompt injection attacks that could streamline malicious use.
Unit 42 researchers also demonstrated effective jailbreaking techniques against DeepSeek R1 and V3, showing that the models are easy to abuse for nefarious purposes.
Google says hackers abuse Gemini AI to empower their attacks
Multiple state-sponsored groups are experimenting with the AI-powered Gemini assistant from Google to increase productivity and to conduct research on potential infrastructure for attacks or for reconnaissance on targets.
Google's Threat Intelligence Group (GTIG) detected government-linked advanced persistent threat (APT) groups using Gemini primarily for productivity gains rather than to develop or conduct novel AI-enabled cyberattacks that can bypass traditional defenses.
Threat actors have been trying to leverage AI tools for their attack purposes to various degrees of success as these utilities can at least shorten the preparation period.
Google has identified Gemini activity associated with APT groups from more than 20 countries but the most prominent ones were from Iran and China.
Among the most common cases were assistance with coding tasks for developing tools and scripts, research on publicly disclosed vulnerabilities, checking on technologies (explanations, translation), finding details on target organizations, and searching for methods to evade detection, escalate privileges, or run internal reconnaissance in a compromised network.
APTs using Gemini
Google says APTs from Iran, China, North Korea, and Russia, have all experimented with Gemini, exploring the tool's potential in helping them discover security gaps, evade detection, and plan their post-compromise activities. These are summarized as follows:
- Iranian threat actors were the heaviest users of Gemini, leveraging it for a wide range of activities, including reconnaissance on defense organizations and international experts, research into publicly known vulnerabilities, development of phishing campaigns, and content creation for influence operations. They also used Gemini for translation and technical explanations related to cybersecurity and military technologies, including unmanned aerial vehicles (UAVs) and missile defense systems.
- China-backed threat actors primarily utilized Gemini for reconnaissance on U.S. military and government organizations, vulnerability research, scripting for lateral movement and privilege escalation, and post-compromise activities such as evading detection and maintaining persistence in networks. They also explored ways to access Microsoft Exchange using password hashes and reverse-engineer security tools like Carbon Black EDR.
- North Korean APTs used Gemini to support multiple phases of the attack lifecycle, including researching free hosting providers, conducting reconnaissance on target organizations, and assisting with malware development and evasion techniques. A significant portion of their activity focused on North Korea's clandestine IT worker scheme, using Gemini to draft job applications, cover letters, and proposals to secure employment at Western companies under false identities.
- Russian threat actors had minimal engagement with Gemini, most usage being focused on scripting assistance, translation, and payload crafting. Their activity included rewriting publicly available malware into different programming languages, adding encryption functionality to malicious code, and understanding how specific pieces of public malware function. The limited use may indicate that Russian actors prefer AI models developed within Russia or are avoiding Western AI platforms for operational security reasons.
Latest
-
Indian tech giant Tata Technologies hit by ransomware attack
Tata Technologies Ltd. had to suspend some of its IT services following a ransomware attack that impacted the company network.
- January 31, 2025
- 11:02 AM
1
-
Globe Life data breach may impact an additional 850,000 clients
Insurance giant Globe Life finished the investigation into the data breach it suffered last June and says that the incident may have impacted an additional 850,000 customers.
- January 31, 2025
- 10:24 AM
- 0
-
Mizuno USA says hackers stayed in its network for two months
Mizuno USA, a subsidiary of Mizuno Corporation, one of the world's largest sporting goods manufacturers, confirmed in data breach notification letters that unknown attackers stole files from its network between August and October 2024.
- January 31, 2025
- 10:12 AM
- 0
Nation-State Hackers Abuse Gemini AI Tool - Infosecurity Magazine
No comments:
Post a Comment