Carnegie Mellon researchers, taking part in DARPA's Mind's Eye program, have created visually intelligent software to recognize human activities in video and then predict what might happen next. If a person holding a gun were to walk up to you, what might you think would happen next? Researchers from Carnegie Mellon have created intelligent software that will identify human activities in videos and then predict what might happen next. It should come as little surprise that the spookily named ‘Mind’s Eye’ program is sponsored by DARPA’s Information Innovation Office. “A truly ‘smart’ camera would be able to describe with words everything it sees and reason about what it cannot see,” said DARPA. Visually intelligent technology previously ‘thought’ in terms of nouns to describe a scene, but Carnegie Mellon researchers have made smart software that can also think in terms of action verbs. “A video shows a woman carrying a box into a building. Later, it shows her leaving the building without it. What was she doing?” asked Carnegie Mellon University.The Mind’s Eye software “will compare the video motion to actions it’s already been trained to recognize (such as walk, jump, and stand) and identify patterns of actions (such as pick up and carry). The software examines these patterns to infer what the person in the video is doing. It also makes predictions about what is likely to happen next and can guess at activities that might be obscured or occur off-camera.”Carnegie Mellon University’s National Robotics Engineering Center explained the image below as: “The Mind’s Eye program will automate video analysis – recognizing current behavior, interpolating actions that occur off-camera, and predicting future behavior.” The next step is to make the ‘Cognitive Engine’ even smarter. According to the report “Using Ontologies in a Cognitive-Grounded System: Automatic Action Recognition in Video Surveillance” [PDF], the researchers “plan to extend the system functionalities in order to support a wider range of action verbs and run tests on a large video dataset.”DARPA explained, “In the first 18 months of the program, Mind’s Eye demonstrated fundamentally new capabilities in visual intelligence, including the ability of automated systems to recognize actions they had never seen, describe observed events using simple text messages, and flag anomalous behaviors.” Carnegie Mellon is one of twelve research teams and three commercial integrators participating in the five-year Mind’s Eye Program. Previously, BRS Labs had the “smartest AI suspicious behavioral recognition system” with “the capability to learn from what it observes, remember activity patterns and adjust to changes in the environment, field of view and equipment – without manual interaction.” Phys reported that the Carnegie Mellon “researchers’ approach is designed to help prevent crimes or dangerous events from happening.” The newest visually intelligent software “system would eventually sound an alarm if it recognized that an action was not permitted, detecting anomalous behaviors. One example of such a scenario would be the cameras at an airport or bus station, flagging a bag abandoned for more than a few minutes.” This Army-funded AI research was disclosed “at the Semantic Technology for Intelligence, Defense, and Security conference at George Mason University.” The Mind’s Eye system could potentially be used to analyze live drone footage. Who knows, it might even be integrated to work with the unblinking surveillance stare of the Army’s 7-story flying football field-sized blimp? It will likely be embraced in the future by the police and by the military to keep soldiers out of harm’s way. It might have home security applications, watching the surveillance video and alerting home owners with a text message before burglars break in.On the creepy privacy invasion side of the coin, it’s one more surveillance technology hunting suspicious behavior. Let’s hope the researchers get it right because when added to social media surveillance helping the government read your mind and future TSA plans to track all ‘daily travels to work, grocery store and social events’, the future surveillance society world could have a very Orwellian no-privacy flavor.Like this? Here’s more posts:Time to disable Java AGAIN: 1 billion at risk from newest critical Java bugFeds Warn of Zombie Apocalypse! Buy emergency kit, but you might be a terrorist if…Famous Patriot Hacktivist The Jester Shares Battle Chest of OSINT ToolsSmartphone snoop: Even when phone sleeps, digital assistant always eavesdrops Facebook Want Button: Collecting massive amounts of data about you has never been easierBusted! Forensic expert who recovered lurid SMS warns: Phone texts don’t die, they hideDeanonymizing You: I know who you are after 1 click online or a mobile callDoes Microsoft Oppose Verizon Spying? Verizon allegedly wants more control over WP8Flame’s vicious sibling miniFlame malware, a cyber-espionage ‘surgical attack tool’Do you allow RDP connections? Cybercrime service sells hacked Fortune 500 accessSurveillance State: From Inside Secret FBI Terrorist Screening Room to TrapWire TrainingSocial media surveillance helps the government read your mindiPredators: Cyberbullying to cyber-harassing troll cop from hell Follow me on Twitter @PrivacyFanatic Related content news Dow Jones watchlist of high-risk businesses, people found on unsecured database A Dow Jones watchlist of 2.4 million at-risk businesses, politicians, and individuals was left unprotected on public cloud server. By Ms. Smith Feb 28, 2019 4 mins Data Breach Hacking Security news Ransomware attacks hit Florida ISP, Australian cardiology group Ransomware attacks might be on the decline, but that doesn't mean we don't have new victims. A Florida ISP and an Australian cardiology group were hit recently. By Ms. Smith Feb 27, 2019 4 mins Ransomware Security news Bare-metal cloud servers vulnerable to Cloudborne flaw Researchers warn that firmware backdoors planted on bare-metal cloud servers could later be exploited to brick a different customer’s server, to steal their data, or for ransomware attacks. By Ms. Smith Feb 26, 2019 3 mins Cloud Computing Security news Meet the man-in-the-room attack: Hackers can invisibly eavesdrop on Bigscreen VR users Flaws in Bigscreen could allow 'invisible Peeping Tom' hackers to eavesdrop on Bigscreen VR users, to discreetly deliver malware payloads, to completely control victims' computers and even to start a worm infection spreading through VR By Ms. Smith Feb 21, 2019 4 mins Hacking Vulnerabilities Security Podcasts Videos Resources Events SUBSCRIBE TO OUR NEWSLETTER From our editors straight to your inbox Get started by entering your email address below. Please enter a valid email address Subscribe