一、常用命令
1.1 Shell
文本处理
获取随机字符串
openssl rand -base64 32
JSON 转 CSV
jq -r '(.[0] | keys_unsorted) as $keys | $keys, map([.[ $keys[] ]])[] | @csv' cat c.txt|jq .data.data | jq -r '(.[0] | keys_unsorted) as $keys | $keys, map([.[ $keys[] ]])[] | @csv' >> 3.csv
awk 类似 mkString 输出
# print 默认输出换行,不想换行可以使用pringf cat data.json | jq -r -c '.data|{f1,f2,f3}'|awk 'BEGIN{printf("%s","[")} {printf("%s%s",NR>1?",":"",$0)}END{printf("%s","]")}
文件处理
任意文件转 base64
file_path=xx.xx file_mime=`file -I ${file_path}|sed 's/^.*://g;s/;.*$//g;s/[ ][ ]*//g'` file_base64=`base64 ${file_path}` js_base64="data:${file_mime};base64,${file_base64}" echo ${js_base64}
批量替换文件后缀
mv /var/lib/containerd/io.containerd.metadata.v1.bolt/meta.db{,.bak}
删除乱码文件
我们在工作的时候由于各种原因会产生乱码文件,产生的乱码的文件主要分为两种
不完全乱码 如:?j?
因为我们可以看到一个字符,可以使用 rm -rvf j,但要注意是否有其他带有关键字的文件完全乱码 如:????
像这样的数据我们就不能使用正则的方法删除文件
# 1. 查看文件inode节点数
ls -i
# 2. 修改乱码文件为a.txt
find ./ -inum 节点数 -exec mv {} a.txt \;
# 3. 删除乱码文件
find ./ -inum 节点数 -exec -print -exec rm -rvf {} \;
例如:
wissy@wissy-PC:/tmp$ ls -i
72294967 ? 69156075 b.txt
wissy@wissy-PC:/tmp$ find ./ -inum 72294967
./?
#删除文件
wissy@wissy-PC:/tmp$ find ./ -inum 72294967 -print -exec rm -rf {} \;
./?
#或者重命名文件
wissy@wissy-PC:/tmp$ find . -inum 69156056 -exec mv {} file.txt \;
网络安全
ACME 更新证书
下载
curl https://get.acme.sh | sh
加载环境变量
source ~/.acme.sh/acme.sh.env 或者 . ~/.acme.sh/acme.sh.env
设置默认签证机构
acme.sh --set-default-ca --server letsencrypt
使用 Aliyun 的域名认证进行签证
export Ali_Key='xxx' export Ali_Secret='xxx' acme.sh --issue -d '*.wissy.com.cn' --dns dns_ali
这些认证信息会保存到: ~/.acme.sh/account.conf 中
设置自动更新证书 cron
"/home/xxx/.acme.sh"/acme.sh --cron --home "/home/xxx/.acme.sh" 或者 "/root/.acme.sh"/acme.sh --cron --home "/root/.acme.sh"
自动更新 acme
acme.sh --upgrade --auto-upgrade
端口映射到本地
autossh
autossh -M 5678 -fCNL 0.0.0.0:80:127.0.0.1:80 -fCNL 0.0.0.0:443:127.0.0.1:443 root@xxx.xx.xxx.x
其他片段
GNU/Linux 一键更换国内软件源脚本
curl -sSL https://gitee.com/SuperManito/LinuxMirrors/raw/main/ChangeMirrors.sh|bash -
问题汇总
yum/dnf
Error: Failed to download metadata for repo 'appstream': Cannot prepare internal mirrorlist: No URLs in mirrorlist
wget 'http://mirror.centos.org/centos/8-stream/BaseOS/x86_64/os/Packages/centos-gpg-keys-8-3.el8.noarch.rpm' sudo rpm -i 'centos-gpg-keys-8-3.el8.noarch.rpm' dnf --disablerepo '*' --enablerepo=extras swap centos-linux-repos centos-stream-repos sudo dnf distro-sync
FYI: StackoverFlow
Npm Permission denied
Npm 执行 npm install hexo-cli --save 过程中报 Permission denied 并且 使用sudo也不能解决
npm config set user 0 npm config set unsafe-perm true
1.2 Mac
常用命令
跳过验证
xattr -d com.apple.quarantine /Applications/xxxx.app
禁用 Chrom 左右双指滑动手势返回上一页
defaults write com.google.Chrome AppleEnableSwipeNavigateWithScrolls -bool false
二、服务配置
2.1 Nginx
Websocket
自动添加 upgrade 标头
map $http_upgrade $connection_upgrade {
default upgrade;
"" close;
}
location 配置
http {
server {
location / {
...
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
}
Vue 单页面配置
http {
server {
#解决Router(mode: 'history')模式下,刷新路由地址不能找到页面的问题
location / {
root /apps/budd/client;
index index.html index.htm;
try_files $uri $uri/ @router; # 配置使用路由
}
# 路由配置信息
location @router {
rewrite ^.*$ /index.html last;
}
...
}
}
Gzip 配置
Gzip_Min_Length: 字面意思,Gzip 压缩的最低大小限制 e.g 1k 1m
Gzip_Comp_Level: Gzip 压缩等级,1~9 级,其中推荐 1 级即可,等级再高效果就不佳了
Gzip_Vary: 是否发送 vary header
http {
...
gzip on;
gzip_min_length 1k;
gzip_buffers 4 16k;
gzip_http_version 1.1;
gzip_comp_level 4;
gzip_types text/plain text/css application/json application/x-javascript text/xml application/xml application/xml+rss text/javascript;
gzip_vary on;
gzip_proxied any;
gzip_disable "MSIE [1-6].";
}
http {
...
gzip on;
gzip_min_length 1k;
gzip_comp_level 1;
gzip_types text/plain application/x-javascript text/css application/xml text/javascript
application/x-httpd-php application/javascript application/json;
gzip_proxied expired no-cache no-store private auth;
gzip_vary on;
}
log_format
log_json
log_format log_json '{"@timestamp": "$time_local", '
'"remote_addr": "$remote_addr", '
'"referer": "$http_referer", '
'"request": "$request", '
'"status": $status, '
'"bytes": $body_bytes_sent, '
'"agent": "$http_user_agent", '
'"upstream_addr": "$upstream_addr",'
'"upstream_status": "$upstream_status",'
'"up_resp_time": "$upstream_response_time",'
'"request_time": "$request_time"'
' }';
2.2 dnsmasq
使用 DNS 屏蔽 AD
## Downloads an extensive list of known ad servers
## saves it at /etc/host.ads, and schedules weekly updates
# URL of ad server list
adserverlist="http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext"
# Ensure that there is a downloaded list available, in case the router recently booted
wget -q -O /etc/hosts.ads "$adserverlist"; service dnsmasq restart
# Schedule automatic weekly updates
echo "@weekly wget -q -O /etc/hosts.ads '$adserverlist'; service dnsmasq restart #WeeklyAdServerListUpdate#" >> /var/spool/cron/crontabs/root
addn-hosts=/etc/hosts.ads
FYI:Blocking ads using dnsmasq with an additional hosts file
AdGurd Hosts
curl https://raw.githubusercontent.com/AdguardTeam/AdguardFilters/master/ChineseFilter/sections/adservers.txt|grep '^||'|awk -F '^' '{print $1}'|sed 's/||/address=\//g'|sed 's/$/\/127.0.0.1/g' > /etc/dnsmasq.d/adhosts
crontab
@weekly wget -q -O /etc/hosts.ads 'http://pgl.yoyo.org/adservers/serverlist.php?hostformat=hosts&showintro=0&mimetype=plaintext';curl https://raw.githubusercontent.com/AdguardTeam/AdguardFilters/master/ChineseFilter/sections/adservers.txt|grep '^||'|awk -F '^' '{print $1}'|sed 's/||/address=\//g'|sed 's/$/\/127.0.0.1/g' > /etc/dnsmasq.d/adhosts; service dnsmasq restart #WeeklyAdServerListUpdate#
/etc/dnsmasq.conf
# Include all files in /etc/dnsmasq.d except RPM backup files
conf-dir=/etc/dnsmasq.d,.rpmnew,.rpmsave,.rpmorig
server=/cn/114.114.114.114
server=/taobao.com/114.114.114.114
server=/taobaocdn.com/114.114.114.114
server=/google.com/223.5.5.5
# AD
address=/ad.youku.com/127.0.0.1
address=/ad.iqiyi.com/127.0.0.1
#Bad
address=/freehao123.com/123.123.123.123
addn-hosts=/etc/hosts.ads
address=/wissy.com.cn/103.222.190.158
address=/raw.githubusercontent.com/151.101.56.133
#log all dns queries
log-queries
log-facility=/var/log/dnsmasq/dnsmasq.log
domain-needed
bogus-priv
#no-resolv
#缓存的数量
cache-size=102400
#如果查询的域名没ttl,则使用此设置为缓存ttl时间
neg-ttl=600
#指定允许返回给客户端最大ttl时间
max-ttl=600
#dnsmasq服务器缓存最大时间设定
max-cache-ttl=3600
#dnsmasq服务器缓存最小时间设定
三、其他
国内加速
Google Fonts 加速代理
https://ajax.googleapis.com =>https://ajax.googleapis.cnpmjs.org
https://fonts.googleapis.com =>https://fonts.googleapis.cnpmjs.org
https://fonts.gstatic.com =>https://fonts.gstatic.cnpmjs.org
https://themes.googleusercontent.com =>https://themes.googleusercontent.cnpmjs.org
如 https://fonts.googleapis.com/css?family=Lato:400,700|Source+Code+Pro:400,500
=>
https://fonts.googleapis.cnpmjs.org/css?family=Lato:400,700|Source+Code+Pro:400,500
Github 加速
评论区