The 6-3 ruling, on Thursday which divided the court along the usual ideological lines, came as the US is still reeling from a spate of mass shootings over the past few weeks, including one in Uvalde, Texas, that left 19 children and two teachers dead a month ago.
The State of New York makes it a crime to possess a firearm without a license, whether inside or outside the home, Xinhua news agency reported.
An individual who wants to carry a firearm outside his home may obtain an unrestricted license to "have and carry" a concealed "pistol or revolver" if he can prove that "proper cause exists" for doing so, according to the law.
"Because the State of New York issues public-carry licenses only when an applicant demonstrates a special need for self-defense, we conclude that the state's licensing regime violates the Constitution," Conservative Justice Clarence Thomas wrote in the ruling.
In a dissent, liberal Justice Stephen Breyer wrote that the court's ruling "severely burdens states' efforts" to curb gun violence.
"The primary difference between the court's view and mine is that I believe the Amendment allows states to take account of the serious problems posed by gun violence," Breyer argued.