On Language models recognize dropout and Gaussian noise applied to their activations

This could be an interesting new paper by Yoshua Bengio and his team!

Caveat: I have not yet read the paper.

From the abstract:

"We provide evidence that language models can detect, localize and, to a certain degree, verbalize the difference between perturbations applied to their activations. More precisely, we either

(a) mask activations, simulating dropout, or

(b) add Gaussian noise to them, at a target sentence.

We then ask a multiple-choice question such as ``Which of the previous sentences was perturbed?'' or ``Which of the two perturbations was applied?''.

We test models from the Llama, Olmo, and Qwen families, with sizes between 8B and 32B, all of which can easily detect and localize the perturbations, often with perfect accuracy. These models can also learn, when taught in context, to distinguish between dropout and Gaussian noise. Notably, qwenb's zero-shot accuracy in identifying which perturbation was applied improves as a function of the perturbation strength and, moreover, decreases if the in-context labels are flipped, suggesting a prior for the correct ones -- even modulo controls.

Because dropout has been used as a training-regularization technique, while Gaussian noise is sometimes added during inference, we discuss the possibility of a data-agnostic ``training awareness'' signal and the implications for AI safety."

 

[2604.17465] Language models recognize dropout and Gaussian noise applied to their activations

Comment on: Attention as an RNN

Very interesting paper by Yoshua Bengio and collaborators!

"The advent of Transformers marked a significant breakthrough in sequence modelling, providing a highly performant architecture capable of leveraging GPU parallelism. However, Transformers are computationally expensive at inference time, limiting their applications, particularly in low-resource settings (e.g., mobile and embedded devices). Addressing this, we (1) begin by showing that attention can be viewed as a special Recurrent Neural Network (RNN) with the ability to compute its many-to-one RNN output efficiently. We then (2) show that popular attention-based models such as Transformers can be viewed as RNN variants. However, unlike traditional RNNs (e.g., LSTMs), these models cannot be updated efficiently with new tokens, an important property in sequence modelling. Tackling this, we (3) introduce a new efficient method of computing attention's many-to-many RNN output based on the parallel prefix scan algorithm. Building on the new attention formulation, we (4) introduce Aaren, an attention-based module that can not only (i) be trained in parallel (like Transformers) but also (ii) be updated efficiently with new tokens, requiring only constant memory for inferences (like traditional RNNs). Empirically, we show Aarens achieve comparable performance to Transformers on 38 datasets spread across four popular sequential problem settings: reinforcement learning, event forecasting, time series classification, and time series forecasting tasks while being more time and memory-efficient."

[2405.13956] Attention as an RNN

Comments on: AI-Assisted Generation of Difficult Math Questions

Food for thought! Would it not be great to use machine learning & AI to develop probing questions about difficult problems as well as solve difficult problems. A different approach for each opposite direction.

Sort of to burn a candle (a very long one) from both ends becomes a new paradigm in AI? 😊 Or like digging a tunnel through a mountain from both sides.

Caveat: I have not read the paper yet.

From the abstract:

"Current LLM training positions mathematical reasoning as a core capability. With publicly available sources fully tapped, there is unmet demand for diverse and challenging mathematics questions. Relying solely on human experts is both time-consuming and costly, while LLM-generated questions often lack the requisite diversity and difficulty. We present a design framework that combines the strengths of LLMs with a human-in-the-loop approach to generate a diverse array of challenging math questions. Initially, leveraging LLM metacognition skills [Didolkar et al., 2024], a strong LLM is used to extract core "skills" from existing math datasets. These skills serve as the basis for generating novel and difficult questions by prompting the LLM with random pairs of core skills that must be utilized in the question. This ``out of distribution'' task is challenging for both LLMs and humans. Our pipeline employs LLMs to iteratively generate and refine questions and solutions through multi-turn prompting. Human annotators then verify and further refine the questions, with their efficiency enhanced through further LLM interactions. Applying this pipeline on skills extracted from MATH dataset [Hendrycks et al., 2021] resulted in a dataset of complex math questions, while improving expert productivity. Despite using skills from the MATH dataset, our approach of combining random skill pairs in questions resulted in noticeably higher quality, as evidenced by:
(a) Lower performance of all models on our questions than on MATH (with open models being the most affected).
(b) Higher performance on MATH when using our questions as in-context examples.
Although focused on mathematics, our methodology seems applicable to other domains requiring structured reasoning. It can be seen as a method for {\em scalable oversight,} where human experts evaluate highly capable AI models by also using AI-assistance."

AI-Assisted Generation of Difficult Math Questions | OpenReview (open access; among the authors are Yoshua Bengio and Sanjeev Arora)

How existential risk became the biggest meme in AI thanks to two Turing Award winners

Very recommendable! The alarmism and hysteria recently raised by two leading AI researchers, i.e. Geoffrey Hinton and Yoshua Bengio, was irresponsible!

"Yann LeCun, a Turing Award winner, ... why he thinks the idea that a superintelligent AI system will take over the world is “preposterously ridiculous.” 

People are worried about AI systems that “are going to be able to recruit all the resources in the world to transform the universe into paper clips,” LeCun said. “That’s just insane.” (He was referring to the “paper clip maximizer problem,” a thought experiment in which an AI asked to make as many paper clips as possible does so in ways that ultimately harms humans, while still fulfilling its main objective.) 

He is in stark opposition to Geoffrey Hinton and Yoshua Bengio, two pioneering AI researchers (and the two other “godfathers of AI”), who shared the Turing prize with LeCun. Both have recently become outspoken about  existential AI risk.

Joelle Pineau, Meta’s vice president of AI research, agrees with LeCun. She calls the conversation ”unhinged.” The extreme focus on future risks does not leave much bandwidth to talk about current AI harms, she says. ...  [The existential-risk crowd] have essentially put an infinite cost on that outcome,” says Pineau. 

“When you put an infinite cost, you can’t have any rational discussions about any other outcomes. And that takes the oxygen out of the room for any other discussion, which I think is too bad.”

While talking about existential risk is a signal that tech people are aware of AI risks, tech doomers have a bigger ulterior motive, LeCun and Pineau say: influencing the laws that govern tech. ..."

How existential risk became the biggest meme in AI

CIFAR & gender equity: All for innovation and technology through gender quotas for equity-deserving groups

Gender quotas are about the opposite of gender equality! Unacceptable! Appalling!

Equity is a term of demagoguery!

In the battle of the sexes, men wake up!!!

A message from Elissa Strome, Executive Director, Pan-Canadian AI Strategy at CIFAR:

"... Setting targets was an important aspect of our strategy. As of this year, CIFAR and our partners at Canada’s National AI Institutes (Amii, Mila and the Vector Institute) have committed to ensuring that a minimum of 25% of Canada CIFAR AI Chairs identify as women or non-binary, and that, over the next five years, at least 40% of new Canada CIFAR AI Chair recruits and at least 30% of their trainees identify as members of equity-deserving groups. We see these targets as minimum starting points, with room for growth in the future. ..."

P.S. CIFAR is a well recognized and leading AI research institute! None other than Yoshua Bengio (a lefty) is the Program Co-Director, Canada CIFAR AI Chair, Learning in Machines & Brains, Pan-Canadian AI Strategy

International Women’s Day 2023: a call for innovation and technology for gender equality – CIFAR

Ms. Gender/Non-binary Quota! 

Yoshua Bengio's blog - first words

Even one of the most celebrated and accomplished research in artificial intelligence (AI) is not necessarily very smart outside the area of his expertise! YB is ideologically biased as if medieval superstition was his preoccupation!

Apparently, YB has created his own blog now and in his presumably first blog post he addresses:
"... If we don’t self-destruct in the meantime. What do I mean and why would we self-destruct? I mean that our society is built on fragile foundations, and most importantly that we are constructing and using tools we invented without the sufficient wisdom required to deploy them in a way that captures well the consequences of our actions." 
Professor Bengio humanity has not self destructed in over 2 million years of existence! Ah this elitist pretense of knowledge again!
"... I believe that our current political-social-economic structures are not appropriate to manage the increased power which science and technology is putting in our hands, endangering the natural equilibrium of this planet"
The socialist in him comes out!
"I have the impression that collectively we are like children playing with nuclear bombs"
His reasoning is infantile and so naive!
"That is why I have decided to devote some of my energy to thinking about the climate crisis, because I believe it is one of the most serious threats to humanity and the planet that our generation and coming generations will have to deal with."
So much for medieval superstition from a Turing Award winner! No, men like him are rather the most serious threat to humanity! And his first blog post goes on and on like this!


I have previously blogged about Yoshua Bengio here:

1. Yoshua Bengio: An Ignorant & Naive Professor From Montreal
2. When NIPS Became NeurIPS

P.S. Apart from YB's political views, I have great respect and admiration for his work as an AI researcher!

Yoshua Bengio's blog - first words - Yoshua Bengio: I often write comments and posts on social media but these tend to be only temporarily visible, so I thought I needed a place to…

When NIPS Became NeurIPS

Posted: 12/4/2018  Updated: 11/24/2020, 12/18/2019

Update Of 11/24/2020

To my horror, I had to discover today for the first time that an older paper from 2001 now has two official URLs because of this ridiculous name change:

1. https://papers.nips.cc/paper/2089-convolution-kernels-for-natural-language
2. http://papers.neurips.cc/paper/2089-convolution-kernels-for-natural-language

Will this nonsense now be applied retroactively to all older papers?

Update Of 12/18/2019

As someone who has never attended any of the NIPS conferences, I was stunned a few days ago when I read in MIT Technology Review. The Download. Your daily dose of what’s up in emerging technology (12/13/2019) following:

“Only two years ago, so I’m told, one of the hottest AI research conferences of the year [NIPS] was more giant party than academic exchange. In a fight for the best talent, companies handed out endless free swag and threw massive, blowout events, including one featuring Flo Rida, hosted by Intel. The attendees—mostly men in their early 20s and 30s—flushed with huge salaries and the giddiness of being highly coveted, drank free booze and bumped the night away. ... Internally, reports of sexual harrassment, anti-Semitism, racism, and ageism, were also driving conference-goers to question whether they should continue to attend.

So when I arrived in 2018, a diversity and inclusion committee had been appointed, and the long-standing abbreviation NIPS was swapped for another one”

Why did the responsible people in charge not simply prohibit or constrain these free booze parties etc. to start with?

NIPS Not Nipples

NIPS (as it was formerly known; Neural Information Systems Processing) has been for decades one of the global, premier conferences on artificial intelligence & machine learning was recently renamed to NeurIPS. The 32nd, first renamed, annual Conference of 2018 (12/2 through 12/8) is ongoing as I write this blog post.

Here are the official reasons for the name change (emphasis added):

1. “The current acronym NIPS has unintended connotations that some members of the community find offensive” (Source 1)
2. “In April, over 120 academics from John Hopkins University in Baltimore, Maryland, signed a letter calling on NIPS to be rebranded following reports of inappropriate behaviour. The letter stated that the "acronym of the conference is prone to unwelcome puns."” (Source 2)

Confusing, the official news release (Source 1) contains a table with polling results (“Do you think we should change the name of the NIPS conference?”), which indicates that a large majority of participants actually disagreed with changing the name (1,218 con answers, 674 pro answers). An astonishing 378 answered they were neutral about the name change (people without a spine?).  

I believe, this renaming is highly immature, childish, and infantile. Does not reflect well on the people responsible for it. Mature people would have disciplined the offenders!

Political Correctness And Its Enablers

Some of the gurus of Artificial Intelligence & Machine Learning, e.g. Yann LeCun & Yoshua Bengio (on the Advisory Board of NIPS in 2018), are hard leftists and feminists (blogged about them here & here). My suspicion is that these and other influential, likeminded gentlemen do not have the spine to admonish anyone in person who makes inappropriate remarks. Thus, these gentlemen prefer to rename the show.

Hope Springs Eternal

One can only hope that some more enlightened people at NIPS will reverse the dim witted decision to rename this conference!

Sources:

1. NIPS Name Change
2. 'NIPS' AI Conference Changes Name, Kind Of

Yoshua Bengio: An Ignorant & Naive Professor From Montreal

Posted: 7/14/2018  Updated: 4/14/2019, 7/15/2018

Update Of 4/14/2019

As of today, Yann LeCun decided to cut me off from commenting on his public posts on his Facebook page. Very fitting for an intolerant leftist full of hubris. He is also one of those hard and naive leftists who can not handle the truth or criticism!

Before I forget, Yann LeCun was just named as one of three AI researchers to win the prestigious Alan Turing Award.

It is his loss not mine! :-)

A Vice President & Chief Scientist At Facebook

Yann LeCun is the VP & Chief AI Scientist at Facebook according to his LinkedIn profile. Yann LeCun is actually one of the superstars of AI & machine learning.

I have recently liked his Facebook page, therefore about two posts by Yann LeCun appeared on my Facebook page in the past few days. Both of them were openly and kind of directly against President Trump. Not only that, he essentially and unreflectively (I hope) regurgitated typical leftist/progressive and absurd talking points like Trump has a mental issue (e.g. wordplay on paranoid and pronoid) or Peter Strzok’s “fiery” response during the Congressional Hearing about him (LeCun later also commented favorably about Strzok’s possibly fake outrage that Trump insulted a Veteran).

Yann LeCun Responded On His Facebook Posting

I had challenged Yann LeCun on both of his recent Facebook postings listed above. Today (7/15/2018) he responded to one of his two Facebook postings:

“You don't need to be "extreme left leaning" to see that Trump is a lying, incompetent racist who has no sense of basic human decency and no respect for democratic institutions. Anyone, conservative or progressive, who is in favor of democracy and human decency can see that. Are you? Can you?” (emphasis added)

My response to his response in return: “Mon dieu, you entertain a narrow, highly distorted and biased view of the current president. I suspect, it has to do with your upbringing in France (myself I am from Germany). I further suspect, you have a very poor and incomplete understanding of U.S. history and the Declaration of Independence as well as the U.S. Constitution/State constitutions. Recherchez vous si'l vous plait. Merci!”

Zuckerberg About An Extremely Left Leaning Place

About April 10, 2018, Mark Zuckerberg, CEO of Facebook, gave a testimony in the U.S. Congress where among other things, he had the courage to admit “I understand where that concern is coming from because Facebook and the tech industry are located in Silicon Valley, which is an extremely left-leaning place” (source; emphasis added). I guess, he knew what he was talking about.

I personally do not like these awful show trials of business leaders in the U.S. Congress at all. It is often not much more than grandstanding and chest beating of our elected politicians to drag a business leader before them and lash out at them. In the case of Mark Zuckerberg this was certainly the case!

The Bubble Of Silicon Valley

They think they are so smart, open minded, and sophisticated, working on cutting/bleeding edge technology to the benefit of all mankind. They are so proud of the wisdom of crowd sourcing etc.

Turns out, that they rather live in a serious bubble and they are mightily trapped/stuck in really narrow minded groupthink!

No Harm Intended

I don’t mean any harm to or to judge Yann LeCun’s political views. Everyone is entitled to their opinions! His professional work in AI is excellent/outstanding and greatly admired by many, including myself.