The complaint charged Ullah with providing material support to a terrorist group, use of a weapon of mass destruction and three bomb-related counts. He did not elaborate.
Mayor Bill de Blasio and Police Commissioner James P O'Neill labeled it an attempted terror attack.
The government said in a statement that "Bangladesh is committed to its declared policy of "zero tolerance" against terrorism, and condemns terrorism and violent extremism in all forms or manifestations anywhere in the world, including Monday morning's incident in New York City".
Ullah is believed to have travelled to Bangladesh in September, returning a month later, CBS says.
The White House has confirmed that US President Donald Trump has been briefed on the situation. David Perdue, of Georgia, both Republicans, have most recently proposed the RAISE Act, which would limit the number of permanent-resident visas and do away with the ability of citizens to bring over relatives other than spouses and minor children.
The man who authorities said had designs of carrying out an ISIS-inspired suicide attack detonated a rudimentary, makeshift bomb strapped to his chest with Velcro and zip ties in one of the New York City subway's busiest underground corridors allegedly told police he carried out the attack because he was upset over USA bombings in territories controlled by the group.
Amazon's Alexa Can Now Wake You Up to Music of Your Choice
In addition, Amazon Music subscribers can choose the song that the music alarm will play through lyrics or by choosing a mood. The new feature will be available for all Alexa-powered devices including the Echo Dot , Echo Show , and Echo Plus .
A US enforcement official familiar with the investigation into Monday's attack said officers had found evidence that Ullah had watched Islamic State propaganda on the internet.
Ullah was taken to a local hospital to be treated for injuries, including burns to his body. Rail and bus services at the nearby Port Authority bus terminal were at a standstill in the immediate aftermath.
Since 1965, America's immigration policy has centered on giving preference to people with advanced education or skills, or people with family ties to USA citizens and, in some cases, legal permanent residents. It consisted of a pipe measuring about a foot long and filled with shrapnel and powder and was detonated by a Christmas ornamental light, a source said. In the last few years, Butrico said, he had some unpleasant encounters with Ullah.
On Monday night, Ullah's family said in statement that it was "heartbroken" and "deeply saddened" by the suffering the attack had caused.
The source said the explosion, if it had detonated properly, "could have inflicted a lot more harm".
The blast took place on an underground walkway connecting subway lines, beneath 42nd Street between Seventh and Eighth avenues at 7.20am. Officers and federal agents also appeared at another Brooklyn address associated with Ullah. Citizens have been able to apply for spouses, parents, children, siblings and the siblings' spouses and minor children; the would-be immigrants are then screened by US officials to determine whether they can come.