Web services: Difference between revisions

From Han Wiki
Jump to navigation Jump to search
add "No space left on device" error on a LAMP web server
mNo edit summary
 
(17 intermediate revisions by the same user not shown)
Line 1: Line 1:
[[Category:System administration]]
= PHP =
= PHP =
== Initial setup of php-fpm ==
{{Testedon|2023-08-25|Ubuntu 22.04}}
Check if max connection value is above 65536: <code>sysctl net.core.somaxconn</code>.  If not, add <code>net.core.somaxconn = 65536</code> at the end of /etc/sysctl.conf file. Then reload using <code>sudo sysctl -p</code>.
Change <code>listen.backlog</code> value to 65536, and <code>listen</code> value to 127.0.0.1:9001.
It's better to utilize TCP socket since it's more scalable. In PHP section for nginx configuration files: <code>fastcgi_pass 127.0.0.1:9001;</code>


== Downgrading PHP 7 to PHP 5.6 ==
== Downgrading PHP 7 to PHP 5.6 ==
Line 9: Line 20:
This may only apply for Ubuntu 14.04.4 or other versions less than 16.04. This is to add Ondřej Surý's PPA repository for PHP. Of course, if you have 14.04 and already have PHP 7.0, you probably have done this. If you don't have add-apt-repository, then please add it by adding a package called <span class="package">python-software-properties</span>.
This may only apply for Ubuntu 14.04.4 or other versions less than 16.04. This is to add Ondřej Surý's PPA repository for PHP. Of course, if you have 14.04 and already have PHP 7.0, you probably have done this. If you don't have add-apt-repository, then please add it by adding a package called <span class="package">python-software-properties</span>.


<source bash="lang">
<syntaxhighlight lang="bash">
$ sudo apt-get install python-software-properties
$ sudo apt-get install python-software-properties
</source>
</syntaxhighlight>


Add repository for PHP
Add repository for PHP


<source lang="bash">
<syntaxhighlight lang="bash">
$ sudo add-apt-repository -y ppa:ondrej/php
$ sudo add-apt-repository -y ppa:ondrej/php
</source>
</syntaxhighlight>


Update package lists:
Update package lists:


<source lang="bash">
<syntaxhighlight lang="bash">
$ sudo apt-get update
$ sudo apt-get update
</source>
</syntaxhighlight>


Install PHP 5.6.  You may also add other extensions you may need for your app (e.g. php5.6-mbstring, php5.6-xml, etc.)
Install PHP 5.6.  You may also add other extensions you may need for your app (e.g. php5.6-mbstring, php5.6-xml, etc.)


<source lang="bash">
<syntaxhighlight lang="bash">
$ sudo apt-get install php5.6
$ sudo apt-get install php5.6
</source>
</syntaxhighlight>


Switch the default PHP to PHP 5.6.
Switch the default PHP to PHP 5.6.


<source lang="bash">
<syntaxhighlight lang="bash">
$ sudo update-alternatives --config php
$ sudo update-alternatives --config php
</source>
</syntaxhighlight>


Reference: [https://by-example.org/ubuntu-16-04-xenial-downgrade-php-7-to-php-5-6/ Downgrade PHP 7 to PHP 5.6]
Reference: [https://by-example.org/ubuntu-16-04-xenial-downgrade-php-7-to-php-5-6/ Downgrade PHP 7 to PHP 5.6]
== enable PHP 5.6 FPM in apache2 ==
<syntaxhighlight lang="bash">
$ a2enmod prox_fcgi setenvif
$ a2enconf php5.6-fpm
</syntaxhighlight>


= SSL/TLS =
= SSL/TLS =
Line 48: Line 66:


1. Create a secure key for CSR
1. Create a secure key for CSR
<source lang="bash" highlight="1">
<syntaxhighlight lang="bash" highlight="1">
$ openssl genrsa -des3 -out server.key 2048
$ openssl genrsa -des3 -out server.key 2048
Generating RSA private key, 2048 bit long modulus
Generating RSA private key, 2048 bit long modulus
Line 56: Line 74:
Enter pass phrase for server.key:
Enter pass phrase for server.key:
Verifying - Enter pass phrase for server.key:
Verifying - Enter pass phrase for server.key:
</source>
</syntaxhighlight>


2. Create an insecure key for CSR sourcing from the secure one
2. Create an insecure key for CSR sourcing from the secure one
<source lang="bash" highlight="1">
<syntaxhighlight lang="bash" highlight="1">
$ openssl rsa -in server.key -out server.key.insecure
$ openssl rsa -in server.key -out server.key.insecure
Enter pass phrase for server.key:
Enter pass phrase for server.key:
writing RSA key
writing RSA key
</source>
</syntaxhighlight>


3. Rename the keys
3. Rename the keys
Line 76: Line 94:
</syntaxhighlight>
</syntaxhighlight>


== Let's Encrypt free SSL certificate ==
or


* ''Last tested on Ubuntu 14.04.2 LTS (trusty) | easy | less than ten minutes | 27 June 2016''
<syntaxhighlight lang="console" highlight="1">
$ openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr
Generating a 2048 bit RSA private key
.............+++
..................+++
writing new private key to 'server.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:New Mexico
Locality Name (eg, city) [Default City]:Albuquerque
Organization Name (eg, company) [Default Company Ltd]:The University of New Mexico
Organizational Unit Name (eg, section) []:IT
Common Name (eg, your name or your server's hostname) []:oraapi02d.unm.edu
Email Address []:


You can find all about '''Let's Encrypt''' initiative at their [https://letsencrypt.org/ website].
Please enter the following 'extra' attributes
The most current instruction can be found at [https://www.eff.org/ EFF]'s [https://certbot.eff.org/ certbot] site.
to be sent with your certificate request
A challenge password []:
An optional company name []:
</syntaxhighlight>


This one is for Ubuntu 14.04 (trusty) and pursues the easier-to-use option. I'm assuming that you have a sudo access, although that isn't an absolute requirement.


=== Installation ===


Download the executable and make it executable.
=== Verifying key and certificate files match ===


<source lang="bash" highlight="1,13">
<syntaxhighlight lang="console">
$ sudo wget https://dl.eff.org/certbot-auto
$ openssl rsa -noout -modulus -in FILE.key
--2016-06-27 18:36:18-- https://dl.eff.org/certbot-auto
$ openssl req -noout -modulus -in FILE.csr
Resolving dl.eff.org (dl.eff.org)... 173.239.79.196
$ openssl x509 -noout -modulus -in FILE.cer
Connecting to dl.eff.org (dl.eff.org)|173.239.79.196|:443... connected.
</syntaxhighlight>
HTTP request sent, awaiting response... 200 OK
Length: 44115 (43K) [text/plain]
Saving to: ‘certbot-auto’


100%[======================================================================================================================================================>] 44,115      --.-K/s  in 0.001s
== Online, automated certificate issuance ==


2016-06-27 18:36:18 (67.8 MB/s) - ‘certbot-auto’ saved [44115/44115]
[[Let's Encrypt]]


$ sudo chmod a+x certbot-auto
[[uacme]]
</source>


I prefer to have this type of executable in <span class="path">/usr/local/bin/</span> folder.  It will make it available for other users as well as make it easy add as a cronjob. The ownership is already correct if you used <code>sudo</code>.
== Creating self-signed certificates (usually for SSL connection) ==


Typing <code>sudo certbot-auto --apache</code> in CLI will get you to an interactive menu that will list out all of your domains on Apache2 and will easily generate certificates and even add those Apache directives in the respective virtual domain configuration files (not 100%, but works most of the time).
<syntaxhighlight lang="bash">
$ sudo a2enmod ssl
$ sudo service apache2 restart


<code>certbot-auto</code> creates a folder in <span class="path">/etc/letsencrypt/</span> as a default.
$ sudo mkdir /etc/apache2/ssl


=== Adding more domains ===
$ sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.pem -outform PEM
</syntaxhighlight>


After the initial installation, if you need to add more domains you can do it directly from the CLI.
= Miscellaneous =


<source lang="bash">
== Check a SSL connection ==
$ sudo certbot-auto run --apache -d mydomain.net
</source>


=== Configuring to auto-renew certificate ===
<syntaxhighlight lang="console">
 
$ openssl s_client -connect test.domain.com:443 -servername test.domain.com
<code>certbot-auto</code> can also auto-renew certificates by adding a command as a cronjob.
</syntaxhighlight>
 
<div class="cli">
0 1,13 * *  * /usr/local/bin/certbot-auto renew --quiet --no-self-upgrade
</div>
 
Use crontab to update the cron jobs, and add the above line.
 
<source lang="bash">
$ sudo crontab -e
</source>
 
[[Category:System administration]]
 
=== Adding multiple domain names for a certificate ===
 
You can use one certificate for multiple domains. The certificate is generated, but you need to install it yourself.


<source lang="bash" highlight="1">
== Browsing on a console ==
$ certbot-auto certonly --webroot -w /srv/www/mysite.com/ -d www.mysite.com -d mysite.com -w /srv/www/blog.mysite.com/ -d blog.mysite.com


IMPORTANT NOTES:
* You can use the text-based web browser w3m to browse the Internet in your console screen. e.g <nowiki>'w3m http://ubuntu.com'</nowiki>
- Congratulations! Your certificate and chain have been saved at
  /etc/letsencrypt/live/mysite.com/fullchain.pem. Your cert
  will expire on 2016-09-26. To obtain a new or tweaked version of
  this certificate in the future, simply run certbot-auto again. To
  non-interactively renew *all* of your certificates, run
  "certbot-auto renew"
- If you like Certbot, please consider supporting our work by:


  Donating to ISRG / Let's Encrypt:  https://letsencrypt.org/donate
== Good resources for setting up mail servers ==
  Donating to EFF:                    https://eff.org/donate-le
</source>


= Miscellaneous =
* [https://scaron.info/blog/debian-mail-postfix-dovecot.html Debian Mail Server with Postfix and Dovecot]
* [https://scaron.info/blog/debian-mail-spf-dkim.html Debian Mail Server, Part II: SPF and DKIM]


== "No space left on device" error on a LAMP web server ==
== "No space left on device" error on a LAMP web server ==
Line 165: Line 176:
If you try to '''create a blank file''',
If you try to '''create a blank file''',


<source lang="bash">
<syntaxhighlight lang="bash">
$ touch forcefsck
$ touch forcefsck
touch: cannot touch 'forcefsck': No space left on device
touch: cannot touch 'forcefsck': No space left on device
</source>
</syntaxhighlight>


you get a report back saying there is no space left on device.  However, when you '''check the disk space''':
you get a report back saying there is no space left on device.  However, when you '''check the disk space''':


<source lang="bash">
<syntaxhighlight lang="bash">
$ df -h
$ df -h
Filesystem      Size  Used Avail Use% Mounted on
Filesystem      Size  Used Avail Use% Mounted on
Line 180: Line 191:
none            5.0M    0  5.0M  0% /run/lock
none            5.0M    0  5.0M  0% /run/lock
none            7.9G  140K  7.9G  1% /run/shm
none            7.9G  140K  7.9G  1% /run/shm
</source>
</syntaxhighlight>


There is still 58% of disk space left, so something else is wrong. After googling about this, it turns out that my inode was running out. To '''check the number of inodes''':
There is still 58% of disk space left, so something else is wrong. After googling about this, it turns out that my inode was running out. To '''check the number of inodes''':


<source lang="bash">
<syntaxhighlight lang="bash">
$ df -i
$ df -i
Filesystem      Inodes  IUsed  IFree IUse% Mounted on
Filesystem      Inodes  IUsed  IFree IUse% Mounted on
Line 192: Line 203:
none          2052885      2 2052883    1% /run/lock
none          2052885      2 2052883    1% /run/lock
none          2052885      47 2052838    1% /run/shm
none          2052885      47 2052838    1% /run/shm
</source>
</syntaxhighlight>


''inode'' stands for index node, which is an index for a file/folder/device/etc. in the Unix file system scheme.
''inode'' stands for index node, which is an index for a file/folder/device/etc. in the Unix file system scheme.
Line 198: Line 209:
To '''find out which folder is causing this massive hemorrhage of inodes''':
To '''find out which folder is causing this massive hemorrhage of inodes''':


<source lang="bash">
<syntaxhighlight lang="bash">
$ sudo -s
$ sudo -s


Line 224: Line 235:
1402
1402
(...)
(...)
</source>
</syntaxhighlight>


It looks like there is a lot of inodes in /var for some reason, now we need to narrow down to a specific directory:
It looks like there is a lot of inodes in /var for some reason, now we need to narrow down to a specific directory:


<source lang="bash">
<syntaxhighlight lang="bash">
$ for i in ./* ; do echo $i; find $i -type f | wc -l; done
$ for i in ./* ; do echo $i; find $i -type f | wc -l; done
(...)
(...)
Line 252: Line 263:
1
1
(...)
(...)
</source>
</syntaxhighlight>


You can check the number of files in any directory by issuing '''ls -l | wc -l''' but I couldn't even do this because there were millions of files that have accumulated over a year. These files had accumulated because PHP isn't doing the garbage collection. Your session.gc_probability may be set to 0. Change it to 1.
You can check the number of files in any directory by issuing '''ls -l | wc -l''' but I couldn't even do this because there were millions of files that have accumulated over a year. These files had accumulated because PHP isn't doing the garbage collection. Your session.gc_probability may be set to 0. Change it to 1.


<source lang="bash">
<syntaxhighlight lang="bash">
$ /usr/lib/php5/maxlifetime
$ /usr/lib/php5/maxlifetime
24
24
</source>
</syntaxhighlight>


It's 24 minutes. Now, here is the command to delete all of the older files.
It's 24 minutes. Now, here is the command to delete all of the older files.


<source lang="bash">
<syntaxhighlight lang="bash">
$ find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec rm {} \;
$ find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec rm {} \;
</source>
</syntaxhighlight>


This isn't necessary if you have the garbage collection enabled from the PHP configuration, but here is a cron job to run every hour as a root if this isn't caused by PHP.
This isn't necessary if you have the garbage collection enabled from the PHP configuration, but here is a cron job to run every hour as a root if this isn't caused by PHP.


<source lang="bash">
<syntaxhighlight lang="bash">
$ crontab -e
$ crontab -e
0      /usr/bin/find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec /bin/rm {} \;
0      /usr/bin/find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec /bin/rm {} \;
</source>
</syntaxhighlight>


===References===
===References===


http://pim.famnit.upr.si/blog/index.php?/archives/172-Running-out-of-inodes,-no-space-left-on-device,-php-not-cleaning-sessions.html (accessed on July 30, 2012)
http://pim.famnit.upr.si/blog/index.php?/archives/172-Running-out-of-inodes,-no-space-left-on-device,-php-not-cleaning-sessions.html (accessed on July 30, 2012)

Latest revision as of 07:54, 31 August 2023

PHP

Initial setup of php-fpm

  • Last tested on Ubuntu 22.04 (2023-08-25)

Check if max connection value is above 65536: sysctl net.core.somaxconn. If not, add net.core.somaxconn = 65536 at the end of /etc/sysctl.conf file. Then reload using sudo sysctl -p.

Change listen.backlog value to 65536, and listen value to 127.0.0.1:9001.

It's better to utilize TCP socket since it's more scalable. In PHP section for nginx configuration files: fastcgi_pass 127.0.0.1:9001;

Downgrading PHP 7 to PHP 5.6

  • Last tested on Ubuntu 14.04.4 LTS (trusty) | easy | less than ten minutes

This should be also applicable for Ubuntu 16.04 (xenial) since it has PHP 7.0 as the default. I had temporarily upgraded to PHP 7.0 to check compatibility of one of our applications and Crypt_RSA package turned out to be the sore spot. So we had to revert back.

This may only apply for Ubuntu 14.04.4 or other versions less than 16.04. This is to add Ondřej Surý's PPA repository for PHP. Of course, if you have 14.04 and already have PHP 7.0, you probably have done this. If you don't have add-apt-repository, then please add it by adding a package called python-software-properties.

$ sudo apt-get install python-software-properties

Add repository for PHP

$ sudo add-apt-repository -y ppa:ondrej/php

Update package lists:

$ sudo apt-get update

Install PHP 5.6. You may also add other extensions you may need for your app (e.g. php5.6-mbstring, php5.6-xml, etc.)

$ sudo apt-get install php5.6

Switch the default PHP to PHP 5.6.

$ sudo update-alternatives --config php

Reference: Downgrade PHP 7 to PHP 5.6

enable PHP 5.6 FPM in apache2

$ a2enmod prox_fcgi setenvif
$ a2enconf php5.6-fpm

SSL/TLS

Generate a CSR

  • Last tested on Ubuntu 14.04.2 LTS (trusty) | easy | less than five minutes

This will generate a 2048-bit key (secure & insecure) and CSR for usage on a website. CSR is short for Certificate Signing Request and is usually requested by CA (Certificate Authority) when trying to obtain a SSL/TLS certificate.

1. Create a secure key for CSR

$ openssl genrsa -des3 -out server.key 2048
Generating RSA private key, 2048 bit long modulus
.....................+++
....................+++
e is 65537 (0x10001)
Enter pass phrase for server.key:
Verifying - Enter pass phrase for server.key:

2. Create an insecure key for CSR sourcing from the secure one

$ openssl rsa -in server.key -out server.key.insecure
Enter pass phrase for server.key:
writing RSA key

3. Rename the keys

$ mv server.key server.key.secure
$ mv server.key.insecure server.key

4. Create the CSR

$ openssl req -new -key server.key -out server.csr

or

 $ openssl req -new -newkey rsa:2048 -nodes -keyout server.key -out server.csr
Generating a 2048 bit RSA private key
.............+++
..................+++
writing new private key to 'server.key'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Country Name (2 letter code) [XX]:US
State or Province Name (full name) []:New Mexico
Locality Name (eg, city) [Default City]:Albuquerque
Organization Name (eg, company) [Default Company Ltd]:The University of New Mexico
Organizational Unit Name (eg, section) []:IT
Common Name (eg, your name or your server's hostname) []:oraapi02d.unm.edu
Email Address []:

Please enter the following 'extra' attributes
to be sent with your certificate request
A challenge password []:
An optional company name []:


Verifying key and certificate files match

$ openssl rsa -noout -modulus -in FILE.key
$ openssl req -noout -modulus -in FILE.csr
$ openssl x509 -noout -modulus -in FILE.cer

Online, automated certificate issuance

Let's Encrypt

uacme

Creating self-signed certificates (usually for SSL connection)

$ sudo a2enmod ssl
$ sudo service apache2 restart

$ sudo mkdir /etc/apache2/ssl

$ sudo openssl req -x509 -nodes -days 365 -newkey rsa:2048 -keyout /etc/apache2/ssl/apache.key -out /etc/apache2/ssl/apache.pem -outform PEM

Miscellaneous

Check a SSL connection

$ openssl s_client -connect test.domain.com:443 -servername test.domain.com

Browsing on a console

  • You can use the text-based web browser w3m to browse the Internet in your console screen. e.g 'w3m http://ubuntu.com'

Good resources for setting up mail servers

"No space left on device" error on a LAMP web server

  • Tested on: Ubuntu 12.04 Precise
  • Difficulty: 2/10
  • Time: >10 minutes + number of files to delete + your WPM

If you try to create a blank file,

$ touch forcefsck
touch: cannot touch 'forcefsck': No space left on device

you get a report back saying there is no space left on device. However, when you check the disk space:

$ df -h
Filesystem      Size  Used Avail Use% Mounted on
/dev/sda1        52G   28G   22G  58% /
udev            7.9G  4.0K  7.9G   1% /dev
tmpfs           3.2G  524K  3.2G   1% /run
none            5.0M     0  5.0M   0% /run/lock
none            7.9G  140K  7.9G   1% /run/shm

There is still 58% of disk space left, so something else is wrong. After googling about this, it turns out that my inode was running out. To check the number of inodes:

$ df -i
Filesystem      Inodes   IUsed   IFree IUse% Mounted on
/dev/sda1      3393040 3393020      20  100% /
udev           2050686     489 2050197    1% /dev
tmpfs          2052885     384 2052501    1% /run
none           2052885       2 2052883    1% /run/lock
none           2052885      47 2052838    1% /run/shm

inode stands for index node, which is an index for a file/folder/device/etc. in the Unix file system scheme.

To find out which folder is causing this massive hemorrhage of inodes:

$ sudo -s

$ cd /

$ for i in /*; do echo $i; find $i -type f | wc -l; done
(...)
/home
34293
/initrd.img
0
/initrd.img.old
0
/lib
14655
/lib64
0
/lost+found
0
/media
0
/mnt
0
/opt
1402
(...)

It looks like there is a lot of inodes in /var for some reason, now we need to narrow down to a specific directory:

$ for i in ./* ; do echo $i; find $i -type f | wc -l; done
(...)
./crash
1
./lib
3186175
./local
0
./lock
0
(...)

$ cd lib

$ for i in ./* ; do echo $i; find $i -type f | wc -l; done
(...)
./pam
6
./php5
3012602
./plymouth
1
(...)

You can check the number of files in any directory by issuing ls -l | wc -l but I couldn't even do this because there were millions of files that have accumulated over a year. These files had accumulated because PHP isn't doing the garbage collection. Your session.gc_probability may be set to 0. Change it to 1.

$ /usr/lib/php5/maxlifetime
24

It's 24 minutes. Now, here is the command to delete all of the older files.

$ find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec rm {} \;

This isn't necessary if you have the garbage collection enabled from the PHP configuration, but here is a cron job to run every hour as a root if this isn't caused by PHP.

$ crontab -e
0      /usr/bin/find /var/lib/php5/ -type f -cmin +$(/usr/lib/php5/maxlifetime) -print0 -exec /bin/rm {} \;

References

http://pim.famnit.upr.si/blog/index.php?/archives/172-Running-out-of-inodes,-no-space-left-on-device,-php-not-cleaning-sessions.html (accessed on July 30, 2012)