In San Francisco, the chairman of that city’s pension fund has put on hold a vote to invest 15% of its assets in hedge funds. In Austin, Texas, officers responsible for the retirement savings of city police officers are discussing whether to withdraw all of their hedge-fund investments. In Harrisburg, Pa., a prominent state official asked the systems that manage money for teachers and other public workers to reconsider the $7.6 billion parked in such investments.

“We need to be talking about this,” Pennsylvania Auditor General Eugene DePasquale said in an interview.

The new conversations were spurred by Calpers, the largest U.S. public pension fund, which last month decided to shed its entire $4 billion in hedge-fund investments over the next year. Calpers said the investments were too small a slice of its $298 billion portfolio to justify the time and expense they required.

So far, those talks haven’t led to widespread exits. But the concerns offer evidence that public pension funds are reconsidering their decadelong pursuit of hedge funds as an alternative way of boosting long-term returns and closing funding gaps.

At stake for hedge funds is billions of dollars in investments made by public pension funds on behalf of public-sector employees. Over the past decade, pensions have increasingly moved away from stocks and bonds and put money into investments such as hedge funds, real estate and private equity as alternatives to stocks and bonds. Now, about half of the U.S. public pensions have some sort of hedge-fund investment, according to data tracker Preqin.

Hedge funds typically bet on and against stocks, bonds or other securities, often using borrowed money. That can amplify their gains and their losses. Hedge funds also charge higher fees than other money managers, usually 2% of assets under management and 20% of profits.

Hedge funds performed better than many investments during the 2008 financial crisis, falling on average by less than 20%, compared with the 37% drop in the S&P 500, according to HFR, a hedge-fund research firm. This helped attract even more pension money. But they have struggled overall to repeat that success in recent years.

Average public-pension gains from hedge funds were 3.6% for the three years ended March 31, according to a review of public pensions with more than $1 billion in assets by Wilshire Trust Universe Comparison Service. That compared with a 10.9% return from private equity, a 10.6% return from stocks and 5.7% from fixed-income investments.

After peaking at 1.81% in 2011, pension allocations to hedge funds fell to 1.35% of total portfolios as of June 30, according to Wilshire.

Calpers wields hefty influence among public pensions, due in large part to its size and history as an early adopter of alternative investments to stocks and bonds. When the pension fund waded into hedge funds 14 years ago, it was among the first to do so.

It is unclear where much of this money could end up, but Calpers officials have said they want to increase their commercial-real-estate investments. Officials last week also said they want to move into other asset-backed investments such as pools of corporate loans. Any moves Calpers makes could portend broader moves from other pension funds.

Even board members in favor of holding firm on their hedge-fund commitments want to know why the California decision shouldn’t change their thinking, according to




        Stephen Nesbitt,

       chief executive of Cliffwater LLC, a consultant to such funds.

“The public funds have got to justify why they are doing it,” said




        Stephen Potter,

       president of asset management for Chicago-based Northern Trust Corp., which advises public pensions and invests their money.

Investors of all types continue to pour money into hedge funds, but there are indications they aren’t quite as enamored with big funds as they once were. The hedge-fund industry managed a record $2.82 trillion at the end of the third quarter, an increase of $18 billion, or 0.6%, from the prior quarter, according to HFR.

Investors added $15.9 billion of new capital to hedge funds in the quarter, a decline from $30.5 billion of new money in the second quarter. And for the first time since 2009, big funds saw less new money than smaller funds. Firms managing less than $1 billion received $5.1 billion in capital inflows, HFR says, while midsize firms, managing between $1 and $5 billion, received $6.6 billion of new cash. The industry’s largest firms, managing greater than $5 billion, received inflows of just $4.2 billion.

Even Calpers had some of the same internal debates about the merits of hedge funds before deciding to exit. In the years leading up to the move there was widespread disagreement about the investments. Some officials pushed to put even more money into hedge funds. Others openly doubted the ability of hedge funds to serve as a buffer during times of stress. “You got eight people in the room and 12 opinions about what to do,” former Chief Investment Officer




        Joseph Dear

       said of hedge funds during a November 2013 discussion, according to an online recording.

One public pension official who took action after Calpers pulled the plug was




        Sampson Jordan,

       chief executive of the Austin Police Retirement System, who began studying whether to exit from the holdings completely. His organization has 15% of its $625 million tied up in hedge funds, and those investments have lost money year to date. “Calpers is like the godfather of retirement systems,” Mr. Jordan said. “It’s always alarming when the big guy makes a statement like that.”

When Rosemary Gannaway, the chairwoman of the County Employees Retirement Fund in Jefferson City, Mo., read about the Calpers decision, she said she also requested that the topic be put on the agenda for her next board meeting. The fund has 25% of its assets in hedge funds, according to Preqin.

—Gregory Zuckerman and Daniel Huang contributed to this article.