Web services: Difference between revisions
change headers |
add "No space left on device" error on a LAMP web server |
||
Line 154: | Line 154: | ||
Donating to EFF: https://eff.org/donate-le | Donating to EFF: https://eff.org/donate-le | ||
</source> | </source> | ||
= Miscellaneous = | |||
== "No space left on device" error on a LAMP web server == | |||
*Tested on: Ubuntu 12.04 Precise | |||
*Difficulty: 2/10 | |||
*Time: >10 minutes + number of files to delete + your WPM | |||
If you try to '''create a blank file''', | |||
<source lang="bash"> | |||
$ touch forcefsck | |||
touch: cannot touch 'forcefsck': No space left on device | |||
</source> | |||
you get a report back saying there is no space left on device. However, when you '''check the disk space''': | |||
<source lang="bash"> | |||
$ df -h | |||
Filesystem Size Used Avail Use% Mounted on | |||
/dev/sda1 52G 28G 22G 58% / | |||
udev 7.9G 4.0K 7.9G 1% /dev | |||
tmpfs 3.2G 524K 3.2G 1% /run | |||
none 5.0M 0 5.0M 0% /run/lock | |||
none 7.9G 140K 7.9G 1% /run/shm | |||
</source> | |||
There is still 58% of disk space left, so something else is wrong. After googling about this, it turns out that my inode was running out. To '''check the number of inodes''': | |||
<source lang="bash"> | |||
$ df -i | |||
Filesystem Inodes IUsed IFree IUse% Mounted on | |||
/dev/sda1 3393040 3393020 20 100% / | |||
udev 2050686 489 2050197 1% /dev | |||
tmpfs 2052885 384 2052501 1% /run | |||
none 2052885 2 2052883 1% /run/lock | |||
none 2052885 47 2052838 1% /run/shm | |||
</source> | |||
''inode'' stands for index node, which is an index for a file/folder/device/etc. in the Unix file system scheme. | |||
To '''find out which folder is causing this massive hemorrhage of inodes''': | |||
<source lang="bash"> | |||
$ sudo -s | |||
$ cd / | |||
$ for i in /*; do echo $i; find $i -type f | wc -l; done | |||
(...) | |||
/home | |||
34293 | |||
/initrd.img | |||
0 | |||
/initrd.img.old | |||
0 | |||
/lib | |||
14655 | |||
/lib64 | |||
0 | |||
/lost+found | |||
0 | |||
/media | |||
0 | |||
/mnt | |||
0 | |||
/opt | |||
1402 | |||
(...) | |||
</source> | |||
It looks like there is a lot of inodes in /var for some reason, now we need to narrow down to a specific directory: | |||
<source lang="bash"> | |||
$ for i in ./* ; do echo $i; find $i -type f | wc -l; done | |||
(...) | |||
./crash | |||
1 | |||
./lib | |||
3186175 | |||
./local | |||
0 | |||
./lock | |||
0 | |||
(...) | |||
$ cd lib | |||
$ for i in ./* ; do echo $i; find $i -type f | wc -l; done | |||
(...) | |||
./pam | |||
6 | |||
./php5 | |||
3012602 | |||
./plymouth | |||
1 | |||
(...) | |||
</source> | |||
You can check the number of files in any directory by issuing '''ls -l | wc -l''' but I couldn't even do this because there were millions of files that have accumulated over a year. These files had accumulated because PHP isn't doing the garbage collection. Your session.gc_probability may be set to 0. Change it to 1. | |||
<source lang="bash"> | |||
$ /usr/lib/php5/maxlifetime | |||
24 | |||
</source> | |||
It's 24 minutes. Now, here is the command to delete all of the older files. | |||
<source lang="bash"> | |||
$ find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec rm {} \; | |||
</source> | |||
This isn't necessary if you have the garbage collection enabled from the PHP configuration, but here is a cron job to run every hour as a root if this isn't caused by PHP. | |||
<source lang="bash"> | |||
$ crontab -e | |||
0 /usr/bin/find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec /bin/rm {} \; | |||
</source> | |||
===References=== | |||
http://pim.famnit.upr.si/blog/index.php?/archives/172-Running-out-of-inodes,-no-space-left-on-device,-php-not-cleaning-sessions.html (accessed on July 30, 2012) |
Revision as of 19:52, 29 June 2016
PHP
Downgrading PHP 7 to PHP 5.6
- Last tested on Ubuntu 14.04.4 LTS (trusty) | easy | less than ten minutes
This should be also applicable for Ubuntu 16.04 (xenial) since it has PHP 7.0 as the default. I had temporarily upgraded to PHP 7.0 to check compatibility of one of our applications and Crypt_RSA package turned out to be the sore spot. So we had to revert back.
This may only apply for Ubuntu 14.04.4 or other versions less than 16.04. This is to add Ondřej Surý's PPA repository for PHP. Of course, if you have 14.04 and already have PHP 7.0, you probably have done this. If you don't have add-apt-repository, then please add it by adding a package called python-software-properties.
$ sudo apt-get install python-software-properties
Add repository for PHP
$ sudo add-apt-repository -y ppa:ondrej/php
Update package lists:
$ sudo apt-get update
Install PHP 5.6. You may also add other extensions you may need for your app (e.g. php5.6-mbstring, php5.6-xml, etc.)
$ sudo apt-get install php5.6
Switch the default PHP to PHP 5.6.
$ sudo update-alternatives --config php
Reference: Downgrade PHP 7 to PHP 5.6
SSL/TLS
Generate a CSR
- Last tested on Ubuntu 14.04.2 LTS (trusty) | easy | less than five minutes
This will generate a 2048-bit key (secure & insecure) and CSR for usage on a website. CSR is short for Certificate Signing Request and is usually requested by CA (Certificate Authority) when trying to obtain a SSL/TLS certificate.
1. Create a secure key for CSR
$ openssl genrsa -des3 -out server.key 2048
Generating RSA private key, 2048 bit long modulus
.....................+++
....................+++
e is 65537 (0x10001)
Enter pass phrase for server.key:
Verifying - Enter pass phrase for server.key:
2. Create an insecure key for CSR sourcing from the secure one
$ openssl rsa -in server.key -out server.key.insecure
Enter pass phrase for server.key:
writing RSA key
3. Rename the keys
$ mv server.key server.key.secure
$ mv server.key.insecure server.key
4. Create the CSR
$ openssl req -new -key server.key -out server.csr
Let's Encrypt free SSL certificate
- Last tested on Ubuntu 14.04.2 LTS (trusty) | easy | less than ten minutes | 27 June 2016
You can find all about Let's Encrypt initiative at their website. The most current instruction can be found at EFF's certbot site.
This one is for Ubuntu 14.04 (trusty) and pursues the easier-to-use option. I'm assuming that you have a sudo access, although that isn't an absolute requirement.
Installation
Download the executable and make it executable.
$ sudo wget https://dl.eff.org/certbot-auto
--2016-06-27 18:36:18-- https://dl.eff.org/certbot-auto
Resolving dl.eff.org (dl.eff.org)... 173.239.79.196
Connecting to dl.eff.org (dl.eff.org)|173.239.79.196|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 44115 (43K) [text/plain]
Saving to: ‘certbot-auto’
100%[======================================================================================================================================================>] 44,115 --.-K/s in 0.001s
2016-06-27 18:36:18 (67.8 MB/s) - ‘certbot-auto’ saved [44115/44115]
$ sudo chmod a+x certbot-auto
I prefer to have this type of executable in /usr/local/bin/ folder. It will make it available for other users as well as make it easy add as a cronjob. The ownership is already correct if you used sudo
.
Typing sudo certbot-auto --apache
in CLI will get you to an interactive menu that will list out all of your domains on Apache2 and will easily generate certificates and even add those Apache directives in the respective virtual domain configuration files (not 100%, but works most of the time).
certbot-auto
creates a folder in /etc/letsencrypt/ as a default.
Adding more domains
After the initial installation, if you need to add more domains you can do it directly from the CLI.
$ sudo certbot-auto run --apache -d mydomain.net
Configuring to auto-renew certificate
certbot-auto
can also auto-renew certificates by adding a command as a cronjob.
0 1,13 * * * /usr/local/bin/certbot-auto renew --quiet --no-self-upgrade
Use crontab to update the cron jobs, and add the above line.
$ sudo crontab -e
Adding multiple domain names for a certificate
You can use one certificate for multiple domains. The certificate is generated, but you need to install it yourself.
$ certbot-auto certonly --webroot -w /srv/www/mysite.com/ -d www.mysite.com -d mysite.com -w /srv/www/blog.mysite.com/ -d blog.mysite.com
IMPORTANT NOTES:
- Congratulations! Your certificate and chain have been saved at
/etc/letsencrypt/live/mysite.com/fullchain.pem. Your cert
will expire on 2016-09-26. To obtain a new or tweaked version of
this certificate in the future, simply run certbot-auto again. To
non-interactively renew *all* of your certificates, run
"certbot-auto renew"
- If you like Certbot, please consider supporting our work by:
Donating to ISRG / Let's Encrypt: https://letsencrypt.org/donate
Donating to EFF: https://eff.org/donate-le
Miscellaneous
"No space left on device" error on a LAMP web server
- Tested on: Ubuntu 12.04 Precise
- Difficulty: 2/10
- Time: >10 minutes + number of files to delete + your WPM
If you try to create a blank file,
$ touch forcefsck
touch: cannot touch 'forcefsck': No space left on device
you get a report back saying there is no space left on device. However, when you check the disk space:
$ df -h
Filesystem Size Used Avail Use% Mounted on
/dev/sda1 52G 28G 22G 58% /
udev 7.9G 4.0K 7.9G 1% /dev
tmpfs 3.2G 524K 3.2G 1% /run
none 5.0M 0 5.0M 0% /run/lock
none 7.9G 140K 7.9G 1% /run/shm
There is still 58% of disk space left, so something else is wrong. After googling about this, it turns out that my inode was running out. To check the number of inodes:
$ df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda1 3393040 3393020 20 100% /
udev 2050686 489 2050197 1% /dev
tmpfs 2052885 384 2052501 1% /run
none 2052885 2 2052883 1% /run/lock
none 2052885 47 2052838 1% /run/shm
inode stands for index node, which is an index for a file/folder/device/etc. in the Unix file system scheme.
To find out which folder is causing this massive hemorrhage of inodes:
$ sudo -s
$ cd /
$ for i in /*; do echo $i; find $i -type f | wc -l; done
(...)
/home
34293
/initrd.img
0
/initrd.img.old
0
/lib
14655
/lib64
0
/lost+found
0
/media
0
/mnt
0
/opt
1402
(...)
It looks like there is a lot of inodes in /var for some reason, now we need to narrow down to a specific directory:
$ for i in ./* ; do echo $i; find $i -type f | wc -l; done
(...)
./crash
1
./lib
3186175
./local
0
./lock
0
(...)
$ cd lib
$ for i in ./* ; do echo $i; find $i -type f | wc -l; done
(...)
./pam
6
./php5
3012602
./plymouth
1
(...)
You can check the number of files in any directory by issuing ls -l | wc -l but I couldn't even do this because there were millions of files that have accumulated over a year. These files had accumulated because PHP isn't doing the garbage collection. Your session.gc_probability may be set to 0. Change it to 1.
$ /usr/lib/php5/maxlifetime
24
It's 24 minutes. Now, here is the command to delete all of the older files.
$ find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec rm {} \;
This isn't necessary if you have the garbage collection enabled from the PHP configuration, but here is a cron job to run every hour as a root if this isn't caused by PHP.
$ crontab -e
0 /usr/bin/find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec /bin/rm {} \;
References
http://pim.famnit.upr.si/blog/index.php?/archives/172-Running-out-of-inodes,-no-space-left-on-device,-php-not-cleaning-sessions.html (accessed on July 30, 2012)