Bristol City lifted the pressure on Millwall and Gary Rowett, its boss, to victory in South London.
In the fifth round, Sheffield United was able to book its place through a dramatic League One Plymouth Argyle showing.
West Ham is easing their way to the fifth round with a thumping win over League One Doncaster.
At Oakwell, the championship leaders were defeated by Barnsley. A single Callum Styles goal provided a home tie with either Chelsea or Luton Town in round five.
Swansea’s 21-year-old midfielder Oli Cooper scored his first senior goal after Jamal Lowe’s initial shot was blocked in a 5-1 win, thumping a shot by Jordan Smith.
Bissouma scored a screamer in a 2-1 win over Blackpool in Brighton.
Swansea 5-1 Nottingham Forest
Barnsley 1-0 Norwich
Millwall 0-3 Bristol City
Brighton 2-1 Blackpool
Sheffield United 2-1 Plymouth
West Ham 4-0 Doncaster