From AdminWiki

(Difference between revisions)

Latest revision as of 12:50, 26 July 2011

General

Setup

Storage

As for every transactional database, disk I/O is the main limiting factor for PostgresSQL. If you plan to deploy a high-usage database please take some precautions on the used storage. Use either a RAID 1 or 10 and choose a FS that does fast block-IO (ext2).

Accounts and permissions

When starting with a fresh database, you've to create some users first before you can start using it.

Basic steps:

su - postgresql
createuser -P <user>
createdb -O <user> <db>

Also be sure to tune the pg_hba.conf in the data-directory to your needs. To emulate MySQLs default "every user needs to authenticate from everywhere"-semantics use the following config:

local   all         all                               md5
host    all         all         127.0.0.1/32          md5
host    all         all         ::1/128               md5

Performance and Tuning

Useful Queries

SELECT * FROM pg_stat_activiy;

This will show you all currently running transactions/queries. Eg. Look out for longer running queries, if things go wrong.

SELECT * FROM pg_lock;

This will show you a list of currently locked resources.

Sequential Scans

If you're getting sequential scans despite having indexes and have run a vacuum analyze recently you might need to increase the default_statistics_target.

max_fsm_*

document me

sort_mem
shared_buffers

Maintenance

Vacuuming

What?

Since PostgresSQL is a transactional database, old rows don't get actually removed/replaced when you update/delete them (since they might be still needed in older/long running transactions). To actually free them you need to issue a vacuum.

A normal vacuum will only mark deprecated rows for reuse, to actually reclaim diskspace (e.g. when having deleted large amounts of data) you need to issue a full vacuum. Please note that it might be faster to backup the data you want to keep and truncate the table if you plan to remove large portions of a table.

autovacuum

Check status of autovacuums in a given Database:

select relname,last_vacuum,last_autovacuum,last_analyze,last_autoanalyze from pg_stat_user_tables;

Transaction ID Wraparound

A regularly run plain or "lazy" vacuum should prevent Transaction ID Wraparound in all cases, a full vacuum is not required.

You can check your TXID counters with the following query:

SELECT datname, age(datfrozenxid) FROM pg_database;

If the age is noticeably higher than 1 Billion (2^30 or 1073741824 to be exact) after a recent vacuum something is probably not working as intended; if it should be anywhere near 2 billion you should take immediate action to prevent problems.

http://www.postgresql.org/docs/current/interactive/maintenance.html#VACUUM-FOR-WRAPAROUND

Fixing broken databases

set zero_damaged_pages to on; vacuum; pray;

Table Size

If you want to know which table in your database claims the most diskspace, here is a query that returns the size of the tables from the current database.

Note: These examples assume that you use the default page size of 8kb.

relfilenode column holds the file name for this table / data. You can find it in the data directory from postgres (main/).
The relkind column holds the type of the data and reltuples the count of rows in this table.

SELECT relname, pg_size_pretty(relpages::bigint * 8 * 1024) as size, relkind, reltuples::bigint as rows, relpages, relfilenode FROM pg_class ORDER BY relpages DESC;

relnames starting with pg_toast are TOAST-storage for large tables. Compare the appended number with the relfilenodes to get the associated table.

If you have a lot of huge data, thre will be a lot of toast tables. With this query you get an additional field that shows the toast table name if this table has one, and the original table for each toast table:

This works until PostgreSQL 9.0, see modified query below.

SELECT relname, pg_size_pretty(relpages::bigint * 8 *1024) AS size, CASE WHEN relkind = 't' THEN (SELECT pgd.relname FROM pg_class pgd WHERE pgd.relfilenode::text = SUBSTRING(pg.relname FROM 10)) ELSE (SELECT pgc.relname FROM pg_class pgc WHERE pg.reltoastrelid = pgc.relfilenode) END AS refrelname, relfilenode, relkind, reltuples::bigint, relpages FROM pg_class pg ORDER BY relpages DESC;

example output:

                 relname                 | relfilenode | relkind |  reltuples  | relpages | relpages_kb
-----------------------------------------+-------------+---------+-------------+----------+-------------
 pg_toast_16496                          |       16499 | t       | 6.74842e+06 |  1684158 |    13473264
 eintrag                                 |       16510 | r       | 3.97601e+06 |   271484 |     2171872
 admin_log                               |       16496 | r       | 9.49351e+06 |   248608 |     1988864
 history                                 |       16654 | r       | 1.98684e+07 |   204714 |     1637712
 ctimes                                  |       16695 | r       | 1.36451e+07 |   189826 |     1518608

For PostgreSQL 9.0 the query below will work, it also adds some additional info for toast indexes:

SELECT pgn.nspname, relname, pg_size_pretty(relpages::bigint * 8 * 1024) AS size, CASE WHEN relkind = 't' THEN (SELECT pgd.relname FROM pg_class pgd WHERE pgd.reltoastrelid = pg.oid) WHEN nspname = 'pg_toast' AND relkind = 'i' THEN (SELECT pgt.relname FROM pg_class pgt WHERE SUBSTRING(pgt.relname FROM 10) = REPLACE(SUBSTRING(pg.relname FROM 10), '_index', '')) ELSE (SELECT pgc.relname FROM pg_class pgc WHERE pg.reltoastrelid = pgc.oid) END::varchar AS refrelname, CASE WHEN nspname = 'pg_toast' AND relkind = 'i' THEN (SELECT pgts.relname FROM pg_class pgts WHERE pgts.reltoastrelid = (SELECT pgt.oid FROM pg_class pgt WHERE SUBSTRING(pgt.relname FROM 10) = REPLACE(SUBSTRING(pg.relname FROM 10), '_index', ''))) END AS relidxrefrelname, relfilenode, relkind, reltuples::bigint, relpages FROM pg_class pg, pg_namespace pgn WHERE pg.relnamespace = pgn.oid AND pgn.nspname NOT IN ('information_schema', 'pg_catalog') ORDER BY relpages DESC;

Sample output:

 nspname  |                           relname                            |    size    |         refrelname         |      relidxrefrelname      | relfilenode | relkind | reltuples | relpages 
----------+--------------------------------------------------------------+------------+----------------------------+----------------------------+-------------+---------+-----------+----------
 pg_toast | pg_toast_12633551                                            | 12 GB      | error_email_collect        |                            |    16113687 | t       |   6506013 |  1532166
 public   | log_email_sent                                               | 4394 MB    | pg_toast_12633624          |                            |    16112041 | r       |  28645328 |   562416
 public   | mail_log                                                     | 3260 MB    | pg_toast_12633661          |                            |    16113649 | r       |  13556149 |   417328
 public   | error_email_collect                                          | 1003 MB    | pg_toast_12633551          |                            |    16113684 | r       |   1161103 |   128345
 public   | log_hash_reference                                           | 789 MB     | pg_toast_12633642          |                            |    18241270 | r       |   7046247 |   100960
 public   | email                                                        | 504 MB     | pg_toast_12633482          |                            |    16111750 | r       |   3673276 |    64568
 public   | log_email_sent_log_id_idx                                    | 492 MB     |                            |                            |    16112048 | i       |  28684712 |    62947
 public   | log_email_sent_pkey                                          | 492 MB     |                            |                            |    16112047 | i       |  28684712 |    62947
 public   | idx_mail_log_date_created                                    | 291 MB     |                            |                            |    16113677 | i       |  13563234 |    37192
 public   | mail_log_pkey                                                | 233 MB     |                            |                            |    16113655 | i       |  13563234 |    29765
 pg_toast | pg_toast_12633551_index                                      | 139 MB     | pg_toast_12633551          | error_email_collect        |    16113689 | i       |   6506013 |    17840

relidxrefrelname is the original table on which the toast table is based for an toast index

More about admin functions regarding databasize and table size can be found in the System Admin Functions under the Table 9-48.

Cancelling queries

To gracefully cancel queries, one can use pg_cancel_backend(). This should be safer than SIGTERMing the according process.

Accessing the Database

psql and other CLI programs

psql password prompts

psql has, unlike the mysql client, no option to supply a password on the command line (which would be insecure). There are two solutions for this problem:

environment variables:

 export PGPASSWORD=password
 export PGUSER=username
 export PGHOST=host

a ~/.pgpass file

pgpass Documentation

Format:

hostname:port:database:username:password

PostgreSQL