带有字符串变量和url中字母增量的CURL请求

Question

这就是我想要实现的目标：

我想根据以下结构返回有效网址列表：

example.com/a
example.com/b
example.com/c

等等一直到

site.com/zzzzzzzzzzzz

我设法检查页面的有效性

curl -Is example.com/a | head -1

当页面不存在时返回 404 错误。

我设法做这样的循环：

for i in {{a..z},{a..z}{a..z},{a..z}{a..z}{a..z},{a..z}{a..z}{a..z}{a..z},{a..z}{a..z}{a..z}{a..z}{a..z},{a..z}{a..z}{a..z}{a..z}{a..z}{a..z},{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z},{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z},{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z},{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}{a..z}}
do
    echo "example.com/"$i 
    curl -Is example.com/$i | head -1
done

测试从 a 到 zzzzzzzzzzz 的所有网址

有没有办法忽略所有404错误并只返回有效的url？

Answer 1

BASE_URL="example.com"

for path in {{-,{a..z}}{-,{a..z}}{-,{a..z}}{-,{a..z}}}
do
    url="$BASE_URL$path"

    STATUS_CODE=$(curl \
        --output /dev/null \
        --silent \
        --write-out "%{http_code}" \
        "$url")

  if (( STATUS_CODE != 404 ))
  then if (( STATUS_CODE = 200 ))
      then
        echo "$url"
      else
        echo STATUS_CODE
      fi
  fi
done