The country's total population stood at 5.26 million as of December last year.
Mr Lee was speaking to Singaporeans at a Mandarin forum hosted by MediaCorp on Singapore's future.
30 Singaporeans took part in the forum. Among them are educators, young parents, seniors and new citizens.
They raised issues ranging from pre-school education to Singapore's low birth rate and population.
The host, veteran presenter Chun Guek Lay, asked what Singapore's ideal population target is if it is to continue its economic progress.
"It's very hard to give a concrete figure, because the situation is evolving. We're gradually increasing our land area, and if we rebuild our older towns, then we can accommodate more people. Today our population is over 5 million. In the future, 6 million or so should not be a problem. Beyond that, we'll have to think more carefully," said Mr Lee.
Participants were polled on whether they think Singaporeans are accepting of foreigners. 77 per cent said 'yes'.
Amongst those who said 'no' was financial consultant Vincent Chua, who said immigrants are in Singapore because there are benefits and they may leave if this changes one day.
"Immigrants are not the only ones who can leave Singapore, because they are not rooted here. Many overseas Singaporeans, they may come home, or they may stay overseas for many years and take up foreign citizenship. We all know this happens. If new immigrants are willing to take root here and be loyal to Singapore, we should treat them graciously," said PM Lee.
Singapore granted some 16,000 citizenships to foreigners in 2011.
You can catch the programme "A Conversation with PM Lee" on Sunday, 23 September at 9pm on Channel 8.
