Hong Kong protesters' challenge: Crowds shrinking, frustration rising.

Hong Kong (CNN) -- Pro-democracy demonstrators in Hong Kong are clinging on to their protest sites in key areas of the tightly packed city, but their numbers are waning along with the patience of some of their fellow citizens.

"At first, I supported them, but then I started to think they are being selfish because they block the roads -- and that's wrong," said Virginia Lai, who has sold newspapers from a stall in the busy district of Mong Kok for 45 years.

Lai says her business is down 30% and getting worse. The student-led demonstrators are camped out at a major intersection in the neighborhood, which witnessed violent clashes between protesters and their opponents over the weekend.

A CNN team at the main protest point in the Admiralty area also witnessed friction Tuesday night as a handful of protesters wearing blue ribbons -- indicating an anti-Occupy stance -- were mobbed by hundreds of Occupy student protesters.

Photos: Hong Kong unrest

Violent outbursts caught on camera

'We are fighting for our future'

The protesters have blocked several main highways in the city for more than a week as they seek to change a decision by China's ruling Communist Party on how the next election for Hong Kong's top leader will work.

At their peak, the demonstrations brought tens of thousands of Hong Kong residents together in spectacular fashion, covering large areas of the semiautonomous territory's central business district in a sea of people.

'Disrupting my life'

But that was last week, when two back-to-back public holidays put work and classes on hold for a lot of people.

As activity has cranked back up in the financial and commercial hub in recent days, crowds at the protest sites have thinned significantly and signs of discontent among other residents have spread.

"I am very angry because this movement is disrupting my life," said Polly Lau, an elderly woman who has lived in Mong Kok all her life. "I think there will be a rebellion actually, a rebellion of the other 7 million people in Hong Kong against them."

The protests have blocked bus and tram routes, worsening traffic and putting more strain on the city's rail network. Some businesses, offices and schools have closed temporarily.

In Tuesday's altercation in Admiralty, a known pro-Beijing activist, Lee See Yin, attempted to address crowds from street level through a megaphone and was surrounded by an angry crowd of hundreds of student protesters who began screaming to drown her out.

She insisted that she was also from Hong Kong and had a right to be heard, asking the crowds, "Is this real democracy?"

The altercation, which involved verbal assault but no apparent physical abuse, lasted 10 to 15 minutes.

Eventually, half a dozen police came over and formed a ring around the handful of anti-Occupy protesters, who then left the area escorted by the officers.

On Sunday, a group of about 30 taxi drivers carried out their own protest to express their frustration with the pro-democracy sit-ins, which they said were affecting their livelihoods, according to local broadcaster RTHK.

Who's powering the Hong Kong protests?

Hong Kong protestors hacked through app

But the protest movement also commands a lot of sympathy among residents of the city, especially after police used tear gas and pepper spray in a failed effort to disperse demonstrators on September 28 -- tactics seen by many as overly harsh.

Is it safe to travel to Hong Kong?

Is it 'deal time'?

Some commentators are arguing that now is a good moment for demonstrators to cash in their chips before they lose too much support.

"The longer this drags on, the more student activists risk looking to average Hongkongers like irritants," wrote William Pesek, an Asia-Pacific columnist for Bloomberg View, suggesting it's now "deal time for Hong Kong's students."

"Why not parlay what's been achieved so far into meaningful concessions from the government?" Pesek said.

His suggestion follows calls from some prominent figures, including the heads of local universities, for students to leave the protest sites for their own safety. Other observers have noted that the demonstrators have succeeded in putting the democracy issue back on the agenda.

But it remains uncertain what kind of deal the protesters might be able to reach with the government. One of the movement's demands has been the resignation of Hong Kong's top leader, Chief Executive C.Y. Leung.

Increasing fatigue

Student leaders and the government agreed late Monday to a framework for formal talks after protesters around the government headquarters gave civil servants better access to the building.

Hong Kong government representative Lau Kong-wah said Tuesday that a final meeting to prepare dialogue between the students and the government would be held Friday in Hong Kong.

It will be an "open meeting" between Hong Kong's Chief Secretary Carrie Lam and protest leaders, he said.

Leung released a video statement saying students should consider the inconveniences to the general public and insisted the students clear the vehicle entrances to the government complex. He also advised students to leave the protest site in Mong Kok.

Some demonstrators have said they are running out of steam after enduring long days and nights camped out on the asphalt amid stifling heat and torrential downpours.

"I'm tired, but I think we have to stay a while longer," said Kristine Wu, a student who has been at the main protest site on Hong Kong Island for a week.

Are you there? Share images, if you can safely

'We really have to stand strong'

Other protesters among the depleted crowd still holding firm at the site Tuesday expressed similar determination to stay put until some kind of result was achieved.

"We really have to stand strong," said Luk Kam Yan, a student who had been protesting for eight days. "There's been a lot of rumors about clearing out, but I feel if we stay here, we still have a bit of bargaining power."

Student leaders have said they will continue the protest until they have productive talks with the government and expressed optimism that their supporters will stick with them.

"Many protesters need rest after nine days of occupation," Lester Shum, the deputy secretary general of the Hong Kong Federation of Students, said Monday. "I don't believe they are already giving up. When they have recovered, they will return."

But the government appears content to watch the demonstrators' numbers dwindle as negotiations drag on.

CNN's Laura Smith-Spark, Esther Pang, Pamela Boykoff, Felicia Wong, Will Ripley, Tim Schwarz, Jane Sit, Nathan Mauger and Karen Smith contributed to this report.