网页抓取 Fbred 时的额外数据

Question

I am trying to webscrape the league table for the EPL but when I do that I am getting duplicate links as well as links to the teams that arent even in the premier league which makes no sense

Here is my output:
['/en/squads/',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/18bb7c10/Arsenal-Stats',
 '/en/squads/8602292d/Aston-Villa-Stats',
 '/en/squads/4ba7cbea/Bournemouth-Stats',
 '/en/squads/cd051869/Brentford-Stats',
 '/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats',
 '/en/squads/cff3d9bb/Chelsea-Stats',
 '/en/squads/47c64c55/Crystal-Palace-Stats',
 '/en/squads/d3fd31cc/Everton-Stats',
 '/en/squads/fd962109/Fulham-Stats',
 '/en/squads/b74092de/Ipswich-Town-Stats',
 '/en/squads/a2d435b3/Leicester-City-Stats',
 '/en/squads/822bd0ba/Liverpool-Stats',
 '/en/squads/b8fd03ef/Manchester-City-Stats',
 '/en/squads/19538871/Manchester-United-Stats',
 '/en/squads/b2b47a98/Newcastle-United-Stats',
 '/en/squads/e4a775cb/Nottingham-Forest-Stats',
 '/en/squads/33c895d4/Southampton-Stats',
 '/en/squads/361ca564/Tottenham-Hotspur-Stats',
 '/en/squads/7c21e445/West-Ham-United-Stats',
 '/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats',
 '/en/squads/',

'/en/squads/b8fd03ef/Manchester-City-Stats', '/en/squads/a6a4e67d/Chelsea-Women-Stats', '/en/squads/53a2f082/皇家马德里统计', '/en/squads/15f49df1/巴塞罗那女子统计', '/en/squads/e2d8892c/巴黎圣日耳曼统计', '/en/squads/7f2012ad/里昂女子统计', '/en/squads/054efa67/拜仁慕尼黑统计', '/en/squads/51ec22be/Bayern-Munich-Women-Stats', '/en/squads/d609edc0/国际米兰统计', '/en/squads/02f8d026/AS-Roma-Stats', '/en/squads/ef99c78c/River-Plate-Stats', '/en/squads/605aca82/Central-Coast-Mariners-Stats', '/en/squads/6082332e/Melbourne-City-Women-Stats', '/en/squads/50f2a074/Red-Bull-Salzburg-Stats', '/en/squads/3d377a8c/St-Polten-Women-Stats', '/en/squads/f1e6c5f1/Club-Brugge-Stats', '/en/squads/231a3620/安德莱赫特女子统计', '/en/squads/6dcc14bf/最强统计', '/en/squads/abdce579/Palmeiras-Stats', '/en/squads/9cdc3b24/Corinthians-Women-Stats', '/en/squads/488c6ba1/Ludogorets-Razgrad-Stats', '/en/squads/b0e76152/Cavalry-FC-Stats', '/en/squads/2ff539f3/Colo-Colo-Stats', '/zh/squads/c48512d3/上海口岸统计', '/en/squads/4c319052/Millonarios-Stats', '/en/squads/edd0d381/萨格勒布迪纳摩统计', '/en/squads/111cbfb1/斯拉维亚-布拉格-统计', '/en/squads/18050b20/FC-哥本哈根-统计', '/en/squads/b00041f6/HB-Koge-Stats', '/en/squads/990519b8/Independiente-del-Valle-Stats', '/en/squads/e6f63673/KuPS-Stats', '/en/squads/2fdb4aef/奥林匹亚科斯统计', '/en/squads/6611f992/Ferencvaros-Stats', '/en/squads/5c7eb1c7/Mohun-Bagan-Stats', '/en/squads/95f42e44/Persepolis-Stats', '/en/squads/66f38ee7/神户胜利船-统计', '/en/squads/c673cb9f/浦和红队-女子统计', '/en/squads/4372a20b/蔚山现代统计', '/en/squads/972e2539/Al-Hilal-Stats', '/en/squads/18d3c3a3/America-Stats', '/en/squads/e334d850/PSV-埃因霍温-统计', '/en/squads/4573a712/Ajax-Women-Stats', '/en/squads/d86248bd/BodoGlimt-Stats', '/en/squads/c1c51bdf/Valerenga-Women-Stats', '/en/squads/ae107695/Libertad-Stats', '/en/squads/8917b8a9/Sporting-Cristal-Stats', '/en/squads/e0b3aa47/RKS-Rakow-Stats', '/en/squads/13dc44fd/Sporting-CP-Stats', '/en/squads/aed59852/FCSB-Stats', '/en/squads/97d80fef/Mamelodi-Sundowns-Stats', '/en/squads/98ce363d/泽尼特统计', '/en/squads/b81aa4fa/凯尔特人统计', '/en/squads/099c6eb5/Red-Star-Belgrade-Stats', '/en/squads/4b682260/Young-Boys-Stats', '/en/squads/e83b563b/Servette-Stats', '/en/squads/f3d8c8b9/马尔默统计', '/en/squads/a2c4c17a/Rosengard-Stats', '/en/squads/ecd11ca2/加拉塔萨雷统计', '/en/squads/c734e22f/基辅迪纳摩-统计', '/en/squads/e2d73ee6/Penarol-Stats', '/en/squads/81d817a3/Los-Angeles-FC-Stats', '/en/squads/2a6178ac/Orlando-Pride-Stats', '/en/squads/7701ed02/Deportivo-Tachira-Stats', '/en/小队/'] ]

不知道为什么最后会有来自随机联赛的球队，也不知道为什么球队会重复自己

这是我的代码：

import requests

from bs4 import BeautifulSoup

url = "https://fbref.com/en/comps/9/Premier-League-Stats"

data = requests.get(url)

scrape = BeautifulSoup(data.text)

leagueTable = scrape.select('table.stats_table')[0]
links = scrape.find_all('a')

hrefs = []
for link in links:
    href = link.get("href")
    if href is not None:  # Causing problems for next for loop with nonType object so need this
        hrefs.append(href)

filtered_hrefs = []
for i in hrefs:
    if '/squads/' in i:
        filtered_hrefs.append(i)

filtered_hrefs

Answer 1

您确实应该使用 selenium 来实现此目的，因为该网站依赖 JavaScript 来呈现内容。

您应该使用浏览器的内置开发工具检查您尝试抓取的页面，以便您可以精确地找到您尝试获取的信息。然后你就可以相应地设计一个 CSS 选择器。

from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By

OPTIONS = webdriver.ChromeOptions()
OPTIONS.add_argument("--headless=true")

URL = "https://fbref.com/en/comps/9/Premier-League-Stats"

with webdriver.Chrome(OPTIONS) as driver:
    driver.get(URL)
    wait = WebDriverWait(driver, 10)
    ec = EC.presence_of_all_elements_located
    selector = By.CSS_SELECTOR, "#results2024-202591_overall tbody tr td[data-stat='team'] a"
    for a in wait.until(ec(selector)):
        print(a.get_attribute("href"))

输出：

https://fbref.com/en/squads/822bd0ba/Liverpool-Stats
https://fbref.com/en/squads/cff3d9bb/Chelsea-Stats
https://fbref.com/en/squads/e4a775cb/Nottingham-Forest-Stats
https://fbref.com/en/squads/18bb7c10/Arsenal-Stats
https://fbref.com/en/squads/b2b47a98/Newcastle-United-Stats
https://fbref.com/en/squads/4ba7cbea/Bournemouth-Stats
https://fbref.com/en/squads/b8fd03ef/Manchester-City-Stats
https://fbref.com/en/squads/fd962109/Fulham-Stats
https://fbref.com/en/squads/8602292d/Aston-Villa-Stats
https://fbref.com/en/squads/d07537b9/Brighton-and-Hove-Albion-Stats
https://fbref.com/en/squads/361ca564/Tottenham-Hotspur-Stats
https://fbref.com/en/squads/cd051869/Brentford-Stats
https://fbref.com/en/squads/7c21e445/West-Ham-United-Stats
https://fbref.com/en/squads/19538871/Manchester-United-Stats
https://fbref.com/en/squads/d3fd31cc/Everton-Stats
https://fbref.com/en/squads/47c64c55/Crystal-Palace-Stats
https://fbref.com/en/squads/8cec06e1/Wolverhampton-Wanderers-Stats
https://fbref.com/en/squads/a2d435b3/Leicester-City-Stats
https://fbref.com/en/squads/b74092de/Ipswich-Town-Stats
https://fbref.com/en/squads/33c895d4/Southampton-Stats

网页抓取 Fbred 时的额外数据

问题描述投票：0回答：1

1个回答

最新问题

网页抓取 Fbred 时的额外数据

问题描述 投票：0回答：1

1个回答

最新问题

问题描述投票：0回答：1