San Francisco Federal Reserve President John Williams said he thinks the central bank should start unwinding its $4.5 trillion balance sheet in the fall.

"My own view is it will be appropriate to start that process this fall," Williams told the Economic Club of Las Vegas on Aug. 2. "At the outset we're going to start really nice and easy." The Federal Open Market Committee in July laid out its planned normalization process, letting its holdings in Treasury securities decline by no more than $6 billion a month, and mortgage-backed securities by $4 billion a month, increasing the caps by $6 billion and $4 billion, respectively, every three months until they reach $30 billion and $20 billion.

Williams said it should take about four years for the Fed's balance sheet to return to a reasonable size. He said the central bank has not yet settled on an ideal size for its balance sheet.

Williams also discussed the Fed's 2% inflation goal, which he expects it to achieve within the next year or two. He said that, while "regular people" may not see much of a difference between 1.5% and 2% inflation, it is important for the Fed to uphold its commitment to other international central banks.

"We've committed to a strategy having inflation around 2% as have almost all of the other major central banks," he said. "We want to make sure that we fulfill that commitment so that we can have a continued trust that we're keeping inflation around our goal."

Williams said the Fed's open discussions about unwinding its balance sheet has not caused strong market reactions because people "understand our plan."

"The fact that we're going to unwind or shrink our balance sheet is the worst-kept secret in the world," he said during a Q&A following his prepared remarks. "We've been talking about it endlessly, incessantly, and what's striking is that the financial markets have not responded at all."