我最近发现了一篇关于如何从 GitHub 中的组织克隆所有存储库的post。最佳答案如下:
gh repo list myorgname --limit 4000 | while read -r repo _; do
gh repo clone "$repo" "$repo"
done
我理解下划线之前和之后的所有内容。是什么意思 _;在上面的命令中?
谢谢!
_
被用作垃圾来自
man bash
(或man -P'less +"/^ *Shell Variables"' bash
):
Shell Variables The following variables are set by the shell: _ At shell startup, set to the pathname used to invoke the shell or shell script being executed as passed in the environment or argument list. Subsequently, expands to the last argument to the previous simple command executed in the foreground, after expansion. Also set to the full pathname used to invoke each command executed and placed in the environment exported to that command. When checking mail, this parameter holds the name of the mail file currently being checked.
此变量将保存上一行的最后一个参数。
我经常用这个:
mkdir -p /some/path/somewhere
cd $_
如果你尝试将它们用作变量,你可能会失望
_='Hello world!'
echo "I said: '$_'"
I said: ''
但在您的示例中,它们被用作垃圾:
因此,
while read -r repo _;...
将通过将第一个(空格分隔)单词存储到$repo
并将其余行存储到(不可用)$_
来解析每一行输入。
示例,显示密码文件中的 GECOS 字段:
while IFS=: read -r user _ uid _ gecos ;do
(( UID == uid )) && echo $user ${gecos%%,*}
done < /etc/passwd
gh repo list
我更喜欢用另一种方式写这个,这将允许使用overall变量来创建统计数据和其他可重用的东西:
declare -i cloned=0 total=0
while read -r repo _; do
total+=1
gh repo clone "$repo" "$repo" &&
cloned+=1
done < <(
gh repo list myorgname --limit 4000
)
printf 'There was %d repo successfully cloned, over %d total\n' \
$cloned $total
您可能会在如何将变量设置为命令的输出
找到更多示例在处理元素之前,您可以将 gh repo list
读取放入数组中:
mapfile -t rList < <(gh repo list myorgname --limit 4000)
declare -i cloned=0 total=${#rList[@]}
printf 'There are %d objects to clone.\n' $total
for repo in "${rList[@]/%$'\11'*}"; do
gh repo clone "$repo" "$repo" &&
cloned+=1
done
printf 'There was %d repo successfully cloned, over %d total\n' \
$cloned $total
例如,您可以计划一次同时下载最多 5 个进程:
gh
克隆完整的演示脚本。(复制自通过同时/并发文件传输加速 rsync?。)
将其另存为
gh_parClone.sh
并赋予他执行权:
#!/bin/bash
maxProc=5
[[ $1 == -v ]] && verbose=1 && shift || verbose=0
orgName="$1"
declare -ai start elap results order
declare -a succeeded failed messages
wait4oneTask() {
printf 'Running task: %d\r' ${#running[@]}
wait -np epid
results[epid]=$?
local _i _line
elap[epid]=" ${EPOCHREALTIME/.} - ${start[epid]} "
if ((results[epid])); then
failed[epid]="${repos[epid]}"
else
succeeded[epid]="${repos[epid]}"
fi
while read -ru "${ghFds[epid]}" -t .02 _line; do
messages[epid]+="$_line"$'\n'
done
exec {ghFds[epid]}<&-
unset "ghFds[epid]"
unset "running[$epid]"
while [[ -v elap[${order[0]}] ]]; do
_i=${order[0]}
printf " - %(%a %d %T)T.%06.0f %-36s %4d %12d\n" "${start[_i]:0:-6}" \
"${start[_i]: -6}" "${repos[_i]}" "${results[_i]}" "${elap[_i]}"
order=("${order[@]:1}")
done
}
mapfile -t rList < <(LANG=C gh repo list "$orgName")
printf 'GH: %d object in \47%s\47. Start clone with max %d instances.\n' \
${#rList[@]} "$orgName" $maxProc
printf " %-22s %-36s %4s %12s\n" Started Repo Rslt 'microseconds'
for repo in "${rList[@]/%$'\11'*}"; do
exec {ghFd}<> <(:)
gh repo clone "$repo"{,} >&$ghFd 2>&1 &
lpid=$!
ghFds[lpid]=$ghFd
repos[lpid]="${repo#$orgName/}"
start[lpid]=${EPOCHREALTIME/.}
running[lpid]=''
order+=("$lpid")
((${#running[@]}>=maxProc)) && wait4oneTask
done
while ((${#running[@]})); do
wait4oneTask
done
printf 'Succeeded: %d.\n%s\n' ${#succeeded[@]} "${succeeded[*]}"
if ((verbose)); then
for pid in "${!succeeded[@]}"; do
printf ' - %s\n' "${succeeded[pid]}"
printf ' %s\n' "${messages[pid]//$'\n'/$'\n' }"
done
fi
printf 'Failed: %d.\n' ${#failed[@]}
if ((${#failed[@]})); then
for pid in "${!failed[@]}"; do
printf ' - %s\n' "${failed[pid]}"
printf ' %s\n' "${messages[pid]//$'\n'/$'\n' }"
done
fi
示例运行:
$ mkdir -p demos/mixxx/foo
$ ./gh_parClone.sh demos
GH: 9 object in 'demos'. Start clone with max 5 instances.
Started Repo Rslt microseconds
- Fri 12 10:41:59.476735 php-logical-filter 0 2100719
- Fri 12 10:41:59.477372 mixxx 1 403229
- Fri 12 10:41:59.477718 Cache 0 2062995
- Fri 12 10:41:59.478078 developer_skins 0 3713021
- Fri 12 10:41:59.478470 Qt-Inspector 0 1949781
- Fri 12 10:41:59.901984 pytaglib 0 2325354
- Fri 12 10:42:01.449372 gmap3 0 2269148
- Fri 12 10:42:01.561883 Motif 0 4105144
- Fri 12 10:42:01.600549 offlineimap 0 4493532
Succeeded: 8.
php-logical-filter Cache developer_skins Qt-Inspector pytaglib gmap3 Motif offlineimap
Failed: 1.
- mixxx
fatal: destination path 'demos/mixxx' already exists and is not an empty directory.
failed to run git: exit status 128
如果使用
-v
标志运行,输出会更长:
$ ./gh_parClone.sh -v demos