A ceremony, attended by Prime Minister Shinzo Abe, is being held at Hiroshima's memorial park before thousands of lanterns are released on the city's Motoyasu river.
The bombing - and a second one on Nagasaki three days later - is credited with bringing to an end World War II.
But it also claimed the lives of at least 140,000 people in the city.
A US B29 bomber called the Enola Gay dropped the uranium bomb, exploding some 600m above the city, at around 08:10 on 6 August 1945.
On that day alone, at least 70,000 people are believed to have been killed. Many more died of horrific injuries caused by radiation poisoning in the days, weeks and months that followed.
Addressing 40,00 people who attended the commemoration ceremony at Hiroshima's peace park near the epicentre of the 1945 attack, Mr Abe called for worldwide nuclear disarmament.
He said that the atomic bomb not only killed thousands of people in Hiroshima but also caused unspeakable suffering to survivors.
"Today Hiroshima has been revived," the prime minister said, "and has become a city of culture and prosperity.
"Seventy years on I want to re-emphasise the necessity of world peace."
Ceremonies began in Hiroshima on Wednesday, with a procession by Buddhist monks and local residents, and a "die-in" by children in front of the Atomic Bomb Dome, one of the few structures left standing so close to the blast's epicentre.
Mr Abe and Hiroshima Mayor Kazumi Matusi are being joined by US Ambassador to Japan Caroline Kennedy for the official ceremony of remembrance on Thursday, which will include silent prayers, the release of doves and a declaration of peace.
Mr Matsui described nuclear weapons as an "absolute evil" while urging the world to put an end to them forever.
"To coexist we must abolish the... ultimate inhumanity that is nuclear weapons. Now is the time to start taking action," he said in his annual speech.
Later in the day, thousands of paper lanterns will be released on the city's Motoyasu River - symbolising the journey to the afterlife of those who died.