Merge branch 'nginx-improvement' into develop

indiachan-spamvector
Thomas Lynch 2 years ago
commit 33daee16db
  1. 194
      INSTALLATION.md
  2. 146
      configs/nginx/nginx.sh

@ -26,8 +26,8 @@
**2. Install dependencies.**
```bash
$ sudo apt-get update
$ sudo apt-get install nginx ffmpeg imagemagick graphicsmagick python-certbot-nginx fonts-dejavu
sudo apt update -y
sudo apt install curl wget nginx ffmpeg imagemagick graphicsmagick python-certbot-nginx fonts-dejavu -y
```
**3. Install MongoDB**
@ -40,67 +40,119 @@ $ sudo apt-get install nginx ffmpeg imagemagick graphicsmagick python-certbot-ng
**5. Install Node.js**
For easy installation, use [node version manager](https://github.com/nvm-sh/nvm) "nvm".
For easy installation, use [node version manager](https://github.com/nvm-sh/nvm#installing-and-updating) "nvm".
Install nvm then run the following commands to get the LTS version of nodejs.
```bash
$ nvm install --lts
nvm install --lts
```
You may install Node.js yourself without nvm if you prefer.
**6. Configure nginx**
**6. (Optional) If you want a .onion address (Tor) and/or .loki address (Lokinet)**
For standard installations, run `configs/nginx/nginx.sh` as root. This will prompt you for installation directory, domains, onion/lokinet, enable geoip, install a letsencrypt certificate with certbot and more.
Install Tor, setup a hidden service, and output your .onion address:
```bash
sudo apt install tor -y
sudo systemctl enable --now tor
sudo sh -c "cat > /etc/tor/torrc" <<'EOF'
HiddenServiceDir /var/lib/tor/jschan/
HiddenServiceVersion 3
HiddenServicePort 80 unix:/var/run/nginx-tor.sock
EOF
For non-standard installations like using a CDN, see [configs/nginx/README.md](configs/nginx/README.md) and DIY.
sudo systemctl restart tor
sudo cat /var/lib/tor/jschan/hostname
```
**7. Get the backend setup & running**
Install Lokinet, setup a SNApp, and find your .loki address:
```bash
sudo curl -so /etc/apt/trusted.gpg.d/oxen.gpg https://deb.oxen.io/pub.gpg
echo "deb https://deb.oxen.io $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/oxen.list
sudo apt update -y
sudo apt install lokinet -y
sudo systemctl enable --now lokinet
sudo sh -c "cat > /etc/tor/torrc" <<'EOF'
[router]
[network]
keyfile=/var/lib/lokinet/snappkey.private
[paths]
[dns]
[bind]
[api]
[bootstrap]
[logging]
EOF
sudo systemctl restart lokinet
nslookup -type=cname localhost.loki # Your loki address is the "canonical name".
```
For Lokinet, make sure to firewall all ports except 80 on `lokitun0` interface. If you use `ufw` for example, you could do:
```bash
# copy example secrets file and edit it to fill out the details
$ cp configs/secrets.js.example configs/secrets.js && editor configs/secrets.js
sudo ufw deny in on lokitun0 to any
sudo ufw allow in on lokitun0 to any port 80 proto tcp
```
Note down the .loki and .onion address for the next step.
# edit rules and faq pages if desired:
$ editor views/custompages/faq.pug views/custompages/rules.pug
# you can also add more .pug files in that folder with the same general format to create other custom pages
**7. Setup nginx**
# install nodejs packages
$ npm install
For standard installations, run `configs/nginx/nginx.sh` as root. This will prompt you for installation directory, domains, onion/lokinet, enable geoip, install a letsencrypt certificate with certbot and more.
# replace the master file for your favicon in gulp/res/icons/master.png
For non-standard installations like using a CDN, see [configs/nginx/README.md](configs/nginx/README.md) and DIY.
# run the setup script. installs pm2 process manager and gulp build system and runs some gulp tasks.
$ npm run-script setup
**8. Get the backend setup & running**
# run gulp reset to setup the database and folder structure and create the admin account. **Default admin account details with random password will be printed to the command line.**
$ gulp reset
# NOTE: dont run gulp reset again unless you want to completely irreversibly wipe everything.
1. Copy the example secrets file and edit it with your mongodb+redis credentials, cookie secrets, etc:
```bash
cp configs/secrets.js.example configs/secrets.js && editor configs/secrets.js
```
# make jschan pm2 a service and load on system startup. this will output some additional commands you need to run to complete the process if you were smart and didnt do everything as "root" user.
$ pm2 startup
2. Edit the rules and FAQ page if desired:
```bash
editor views/custompages/faq.pug views/custompages/rules.pug
```
# save the process list so jschan is started with pm2
$ pm2 save
3. Install nodejs dependencies:
```bash
npm install
```
# start all the backend processes
$ npm run-script start
$ gulp
4. Replace `gulp/res/icons/master.png` with your desired favicon image.
# some commands you may need to use in future/find helpful
$ pm2 list #list running pm2 processes
$ pm2 logs #see logs
$ pm2 reload ecosystem.config.js #reload all backend processes
5. Run the setup script. This will install `pm2` (nodejs process manager) and `gulp` (task system) and runs some gulp tasks.
```bash
npm run-script setup
```
# gulp is used for various jobs like minifying and compiling scripts
# the build-worker process may also run some of these for certain operations e.g. editing global settings in the web panel
$ gulp --tasks #list available gulp tasks
$ gulp migrate #check for and run db migrations
$ gulp password #reset the admin account password if you forgot it
$ gulp #run default gulp task
6. Run gulp reset to initialize the database and folder structure, and create the admin account. **Default admin account with random password will be printed to the command line.** NOTE: dont run gulp reset again unless you want to completely irreversibly wipe everything.
```bash
gulp reset
```
**8. Optionally, if you plan to use the webring and want to make requests with a proxy to mask your origin server IP:**
7. Make pm2 a system service and load on system startup. NOTE: This will also output some additional commands you need to run to complete the process. Read the command output carefully.
```
pm2 startup
pm2 save
```
8. Start all the backend processes
```bash
npm run-script start
gulp
```
Some commands you may need to use in future/find helpful:
- `pm2 list` - Lists running pm2 processes.
- `pm2 logs` - Start tailing logs.
- `pm2 reload ecosystem.config.js` - Reload all backend processes.
- `gulp password` - Reset the admin account password if you forgot it.
- `gulp` - Run the default gulp task.
More information and commands for customisation or advanced use is in the ADVANCED section.
**9. (Optionall) if you plan to use the webring and want to make requests with a proxy to mask your origin server IP**
EITHER:
@ -112,44 +164,64 @@ Either of the first two options will allow you to follow .onions in your webring
To enable the proxy, tick "Use Socks Proxy" in global management settings and set the appropriate proxy address, e.g. `socks5h://127.0.0.1:9050`, then save settings.
**10. All done!**
## Updating
0. `cd` to your jschan installation folder.
1. Stop the running backend:
```bash
#first, stop the jschan backend
$ pm2 stop ecosystem.config.js
pm2 stop ecosystem.config.js
```
#update node.js to the latest LTS node version, latest npm, and reinstall global packages:
#NOTE: Only works if you installed node.js "nvm" as per the recommendation in the installation instructions.
$ nvm install --lts --reinstall-packages-from=$(node --version) --latest-npm
2. (Optional) Update node.js to the latest LTS node version, latest npm, and reinstall global packages. NOTE: Only works if you installed node.js "nvm" as per the recommendation in the installation instructions.
```bash
nvm install --lts --reinstall-packages-from=$(node --version) --latest-npm
```
#pull the latest changes
$ git pull
3. Pull the latest changes from git:
```bash
git pull
```
#install dependencies again in case any have updated or changed
$ npm install
4. Install nodejs dependencies again in case any have updated or changed
```bash
npm install
```
#check if anything nginx related changed between the old and new verison, e.g.
$ git diff v0.1.5 v0.1.6 configs/nginx
#If you use a completely standard jschan nginx, run configs/nginx/nginx.sh again.
#Otherwise, update your nginx config with the necessary changes.
5. Check if the nginx config needs updating, comparing the version you updated from with the current version:
```bash
git diff v0.9.2 v0.8.0 configs/nginx
```
If the output was blank, goto step 6.
#run the gulp migrate task. this will update things such as your database schema.
$ gulp migrate
#run the default gulp task to update, scripts, css, icons, images and delete old html
$ gulp
If the output showed changes and you used `configs/nginx/nginx.sh` to setup nginx, run it again to reconfigure nginx and overwrite your old configuration.
#start the backend again
$ pm2 restart ecosystem.config.js --env production
If you have a non-standard nginx config, update your nginx config yourself.
#if something breaks, check and read the logs, they will help figure out what went wrong
$ pm2 logs
6. Run the gulp migrate task. This will run "migrations" such as updating the database schema or file structure:
```bash
gulp migrate && gulp
```
7. Start the backend:
```bash
pm2 restart ecosystem.config.js --env production
```
At this point, your installation is updated. If something is broken, check and read the logs, they will help figure out what went wrong:
```bash
pm2 logs
```
## Help! Something didn't work!!!1!!1
If you are sure you did everything correctly and you still can't get it working, you can ask for help in the IRC (linked in [README](README.md)).
Be polite, ask [smart questions](http://www.catb.org/~esr/faqs/smart-questions.html), and keep in mind nobody is obliged to help you.
Be polite, be patient, ask [smart questions](http://www.catb.org/~esr/faqs/smart-questions.html), and keep in mind nobody is obliged to help you.
Paid support is available at a rate of $50USD/hr, payable in cryptocurrency (BTC/XMR) only. Email the address on my [Gitgud profile](https://gitgud.io/fatchan) to inquire.
## Advanced

@ -5,17 +5,27 @@
[[ "$EUID" -ne 0 ]] && echo "Please run as root" && exit;
echo "[jschan nginx configuration helper]"
read -p "Enter the directory you cloned jschan (blank=$(pwd)): " JSCHAN_DIRECTORY
read -p "Enter the directory you cloned jschan, no trailing slash. (blank=$(pwd)): " JSCHAN_DIRECTORY
JSCHAN_DIRECTORY=${JSCHAN_DIRECTORY:-$(pwd)}
read -p "Enter your clearnet domain name e.g. example.com (blank=no clearnet domain): " CLEARNET_DOMAIN
SITES_AVAILABLE_NAME=${CLEARNET_DOMAIN:-jschan} #not sure on a good default, used for sites-available config name
read -p "Enter tor .onion address (blank=no .onion address): " ONION_DOMAIN
read -p "Enter lokinet .loki address (blank=no .loki address): " LOKI_DOMAIN
read -p "Would you like to add a www. subdomain? (y/n): " ADD_WWW_SUBDOMAIN
if [ "$CLEARNET_DOMAIN" != "" ]; then
read -p "Run certbot and automatically configure a certificate for https on clearnet? (y/n): " CERTBOT
if [ "$CERTBOT" == "n" ]; then
read -p "Generate a self-signed certificate instead? (y/n): " SELFSIGNED
fi
if [ "$SELFSIGNED" == "n" ]; then
read -p "Warning: no https certificate chosen for clearnet. Continue without https? (y/n): " NOHTTPS
[[ "$NOHTTPS" == "n" ]] && echo "Exiting..." && exit;
fi
fi
read -p "Should robots.txt disallow compliant crawlers? (y/n): " ROBOTS_TXT_DISALLOW
read -p "Allow google captcha in content-security policy? (y/n): " GOOGLE_CAPTCHA
read -p "Allow Hcaptcha in content-security policy? (y/n): " H_CAPTCHA
read -p "Download and setup geoip for post flags? (y/n): " GEOIP
read -p "Use certbot to install letsencrypt certificate for https? (y/n): " LETSENCRYPT
#looks good?
read -p "Is this correct?
@ -23,6 +33,10 @@ jschan directory: $JSCHAN_DIRECTORY
clearnet domain: $CLEARNET_DOMAIN
.onion address: $ONION_DOMAIN
.loki address: $LOKI_DOMAIN
www subdomains: $ADD_WWW_SUBDOMAIN
certbot https cert: $CERTBOT
self-signed https cert: $SELFSIGNED
no https cert: $NOHTTPS
robots.txt disallow all: $ROBOTS_TXT_DISALLOW
google captcha: $GOOGLE_CAPTCHA
hcaptcha: $H_CAPTCHA
@ -35,8 +49,25 @@ geoip: $GEOIP
if [[ -f /etc/nginx/sites-available/$SITES_AVAILABLE_NAME ]]; then
read -p "/etc/nginx/sites-available/$SITES_AVAILABLE_NAME already exists. Continue and overwrite existing configuration? (y/n)" OVERWRITE
[[ "$OVERWRITE" == "n" ]] && echo "Exiting..." && exit;
rm /etc/nginx/sites-available/$SITES_AVAILABLE_NAME
rm /etc/nginx/sites-enabled/$SITES_AVAILABLE_NAME
fi
echo "Stopping nginx..."
sudo systemctl stop nginx
if [ "$CERTBOT" == "y" ]; then
#run certbot for certificate
if [ "$ADD_WWW_SUBDOMAIN" == "y" ]; then
echo "Running certbot to setup SSL cert for $CLEARNET_DOMAIN and www.$CLEARNET_COMAIN..."
sudo certbot certonly --nginx -d $CLEARNET_DOMAIN -d www.$CLEARNET_DOMAIN
else
echo "Running certbot to setup SSL cert for $CLEARNET_DOMAIN..."
sudo certbot certonly --nginx -d $CLEARNET_DOMAIN
fi
fi
echo "Copying snippets to nginx folder & replacing paths..."
#copy the snippets and replace install path, they aren't templated
sudo cp $JSCHAN_DIRECTORY/configs/nginx/snippets/* /etc/nginx/snippets
sudo sed -i "s|/path/to/jschan|$JSCHAN_DIRECTORY|g" /etc/nginx/snippets/*
@ -46,14 +77,72 @@ JSCHAN_CONFIG="upstream chan {
server 127.0.0.1:7000;
}"
#Use some variabels to make the templating easier later, depending on if they want www. or not
CLEARNET_SERVER_NAME="$CLEARNET_DOMAIN"
LOKI_SERVER_NAME="$LOKI_DOMAIN"
ONION_SERVER_NAME="$ONION_DOMAIN"
if [ "$ADD_WWW_SUBDOMAIN" == "y" ]; then
CLEARNET_SERVER_NAME="$CLEARNET_DOMAIN www.$CLEARNET_DOMAIN"
LOKI_SERVER_NAME="$LOKI_DOMAIN www.$LOKI_DOMAIN"
ONION_SERVER_NAME="$ONION_DOMAIN www.$ONION_DOMAIN"
fi
if [ "$CLEARNET_DOMAIN" != "" ]; then
if [ "$LETSENCRYPT" == "y" ]; then
#run certbot for certificate
sudo certbot certonly --standalone -d $CLEARNET_DOMAIN -d www.$CLEARNET_DOMAIN
HTTPS_MIDSECTION=""
HTTPS_CERT_SECTION=""
if [ "$CERTBOT" == "y" ]; then
HTTPS_CERT_SECTION="
ssl_certificate /etc/letsencrypt/live/$CLEARNET_DOMAIN/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/$CLEARNET_DOMAIN/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
"
elif [ "$SELFSIGNED" == "y" ]; then
echo "Generating self-signed SSL cert..."
mkdir /etc/ssl/private/
mkdir /etc/ssl/certs/
sudo openssl req -x509 -nodes -days 3650 -newkey rsa:2048 -keyout /etc/ssl/private/nginx-selfsigned.key -out /etc/ssl/certs/nginx-selfsigned.crt
echo "Generating dh group. This may take a while..."
sudo openssl dhparam -out /etc/nginx/dhparam.pem 4096
cat > /etc/nginx/snippets/ssl-params.conf <<EOF
ssl_session_cache shared:le_nginx_SSL:10m;
ssl_session_timeout 1440m;
ssl_session_tickets off;
ssl_protocols TLSv1.2 TLSv1.3;
ssl_prefer_server_ciphers on;
ssl_ciphers "ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-RSA-AES128-SHA";
EOF
HTTPS_CERT_SECTION="
ssl_certificate /etc/ssl/certs/nginx-selfsigned.crt;
ssl_certificate_key /etc/ssl/private/nginx-selfsigned.key;
ssl_dhparam /etc/nginx/dhparam.pem;
include /etc/nginx/snippets/ssl-params.conf;
"
fi
#onion_location rediret header
HTTPS_MIDSECTION="
listen [::]:443 ssl ipv6only=on;
listen 443 ssl;
$HTTPS_CERT_SECTION
}
server {
if (\$host = www.$CLEARNET_DOMAIN) {
return 301 https://\$host\$request_uri;
}
if (\$host = $CLEARNET_DOMAIN) {
return 301 https://\$host\$request_uri;
}
server_name $CLEARNET_SERVER_NAME;
return 444;
"
#onion_location redirect header
ONION_LOCATION=""
if [ "$ONION_DOMAIN" != "" ]; then
ONION_LOCATION="add_header onion-location 'http://$ONION_DOMAIN\$request_uri';"
@ -63,41 +152,24 @@ if [ "$CLEARNET_DOMAIN" != "" ]; then
JSCHAN_CONFIG="${JSCHAN_CONFIG}
server {
server_name www.$CLEARNET_DOMAIN $CLEARNET_DOMAIN;
server_name $CLEARNET_SERVER_NAME;
client_max_body_size 0;
$ONION_LOCATION
listen [::]:443 ssl ipv6only=on; # managed by Certbot
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/$CLEARNET_DOMAIN/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/$CLEARNET_DOMAIN/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
include /etc/nginx/snippets/security_headers.conf;
include /etc/nginx/snippets/error_pages.conf;
include /etc/nginx/snippets/jschan_clearnet_routes.conf;
include /etc/nginx/snippets/jschan_common_routes.conf;
}
server {
if (\$host = www.$CLEARNET_DOMAIN) {
return 301 https://\$host\$request_uri;
} # managed by Certbot
if (\$host = $CLEARNET_DOMAIN) {
return 301 https://$host\$request_uri;
} # managed by Certbot
server_name www.$CLEARNET_DOMAIN $CLEARNET_DOMAIN;
$HTTPS_MIDSECTION
listen 80;
listen [::]:80;
return 444; # managed by Certbot
}"
#replace clearnet domain in snippets
echo "Replacing clearnet domain in snippets..."
sudo sed -i "s/example.com/$CLEARNET_DOMAIN/g" /etc/nginx/snippets/*
fi
@ -122,10 +194,12 @@ server {
}"
#replace onion domain in snippets
echo "Replacing onion domain in snippets..."
sudo sed -i "s/example.onion/$ONION_DOMAIN/g" /etc/nginx/snippets/*
else
#no onion, remove it from CSP
echo "No onion, removing example.onion from CSP..."
sudo sed -i 's/ wss:\/\/www.example.onion\/ wss:\/\/example.onion\///g' /etc/nginx/snippets/security_headers*
fi
@ -148,24 +222,34 @@ server {
}"
#replace lokinet domain in snippets
echo "Replacing loki domain in snippets..."
sudo sed -i "s/example.loki/$LOKI_DOMAIN/g" /etc/nginx/snippets/*
else
#no lokinet, remove it from csp
echo "No lokinet, removing example.loki from CSP..."
sudo sed -i 's/ wss:\/\/www.example.loki\/ wss:\/\/example.loki\///g' /etc/nginx/snippets/security_headers*
fi
#write the config to file and syymlink to sites-available
echo "Writing main jschan vhost config..."
printf "$JSCHAN_CONFIG" > /etc/nginx/sites-available/$SITES_AVAILABLE_NAME
sudo ln -s -f /etc/nginx/sites-available/$SITES_AVAILABLE_NAME /etc/nginx/sites-enabled/$SITES_AVAILABLE_NAME
if [ "$NOHTTPS" == "Y" ]; then
echo "Adjusting config snippets to support NOHTTPS mode..."
sudo sed -i "s/Forwarded-Proto https/Forwarded-Proto http/g" /etc/nginx/snippets/jschan_clearnet_routes.conf
fi
if [ "$GOOGLE_CAPTCHA" == "y" ]; then
echo "Allowing recaptcha in CSP..."
#add google captcha CSP exceptions
sudo sed -i "s|script-src|script-src https://www.google.com/recaptcha/ https://www.gstatic.com/recaptcha/ |g" /etc/nginx/snippets/*
sudo sed -i "s|frame-src|frame-src https://www.google.com/recaptcha/ https://recaptcha.google.com/recaptcha/ |g" /etc/nginx/snippets/*
fi
if [ "$H_CAPTCHA" == "y" ]; then
echo "Allowing hcaptcha in CSP..."
#add hcaptcha CSP exceptions
sudo sed -i "s|script-src|script-src https://hcaptcha.com https://*.hcaptcha.com |g" /etc/nginx/snippets/*
sudo sed -i "s|frame-src|frame-src https://hcaptcha.com https://*.hcaptcha.com |g" /etc/nginx/snippets/*
@ -174,12 +258,13 @@ if [ "$H_CAPTCHA" == "y" ]; then
fi
if [ "$ROBOTS_TXT_DISALLOW" == "y" ]; then
echo "Setting robots.txt to disallow all..."
#add path / (all) to disallow to make robots.txt block all robots instead of allowing
sudo sed -i "s|Disallow:|Disallow: /|g" /etc/nginx/snippets/jschan_common_routes.conf
fi
if [ "$GEOIP" == "y" ]; then
echo "Downloading and installing geoip database for nginx..."
#download geoip data
cd /usr/share/GeoIP
mv GeoIP.dat GeoIP.dat.bak
@ -189,14 +274,17 @@ if [ "$GEOIP" == "y" ]; then
chown www-data:www-data /usr/share/GeoIP/GeoIP.dat
#add goeip_country to /etc/nginx/nginx.conf, only if not already exists
grep -qF "geoip_country /usr/share/GeoIP/GeoIP.dat;" /etc/nginx/nginx.conf
if [ $? -eq 0 ]; then
grep -qF "geoip_country" /etc/nginx/nginx.conf
if [ $? -eq 1 ]; then
sudo sed -i '/http {/a \
geoip_country /usr/share/GeoIP/GeoIP.dat;' /etc/nginx/nginx.conf
fi
else
echo "Geoip not installed, removing directives..."
sudo sed '/geoip_country/d' /etc/nginx/nginx.conf
sudo sed '/geoip_country_code/d' /etc/nginx/snippets/jschan_clearnet_routes.conf
fi
echo "Restarting nginx..."
#and restart nginx
sudo systemctl restart nginx

Loading…
Cancel
Save