Footage showed tens of thousands of soldiers with their heads bowed as a giant portrait of Mr Kim was carried slowly through the streets.
His successor and third son, Kim Jong-un, walked beside the hearse, images from state television showed.
Kim Jong-il died of a heart attack on 17 December, aged 69, state media said.
He has been lying in state for the past 10 days.
No schedule was released ahead of the commemorations and no foreign delegations are attending.
But observers said the ceremonies echoed the displays of pomp and military might that marked the death of Mr Kim's father, Kim Il-sung, in 1994.
Kim Jong-un - who is thought to be in his late 20s and who has little political experience - was shown weeping beside the hearse as it drove through the snowy capital.
He was accompanied by his uncle, Chang Song-taek, who is expected to be a key player as the younger Kim consolidates power.
Ri Yong-ho, the army chief, also accompanied the hearse as it drove past ranks of troops.
The three-hour funeral procession was led by a limousine bearing a huge portrait of a smiling Kim Jong-il. The coffin was draped in a red flag and surrounded by white flowers.
As it passed by, crowds of mourners wailed and flailed their arms as soldiers struggled to keep them from spilling into the road.
One soldier interviewed by North Korean state television said: "The snow is endlessly falling like tears. How could the sky not cry when we've lost our general who was a great man from the sky? As we're separated from the general by death, people, mountains and sky are all shedding tears of blood. Dear Supreme Commander!"
The procession was broadcast live on state television. When it ended outside Pyongyang's Kumsusan Memorial Palace, state
TV began broadcasting documentaries about Kim Jong-il's life.
Mr Kim's body had previously lain in state in a glass coffin at the palace.
Observers are keenly watching the line-up over the two-day funeral to see which officials are in prominent positions.
Jostling for influence?
Kim Jong-il - known in North Korea as the "Dear Leader" - was in the process of formalising Kim Jong-un as his successor when he died. However, the transition was not complete, leaving regional neighbours fearful of a power struggle in the nuclear-armed pariah state.
The BBC's Lucy Williamson, in the South Korean capital Seoul, says senior military and party officials may well now be jostling for influence in the new regime.
North Korea's reluctance to open up the funeral ceremony to foreign delegations may signal that those hierarchies have not yet been fully agreed, she adds.
In the week since Mr Kim died, state media has called Kim Jong-un the "Great Successor" and referred to him as the leader of the military and the party.
Commemorations are expected to continue on Thursday, with a three-minute silence at noon local time (03:00 GMT), followed by trains and ships sounding horns. The national memorial service will then begin.
The inter-Korean Kaesong industrial park has been closed for two days for the mourning following a North Korean request, South Korea's Yonhap news agency reports.