GIDS 2015, Hyderabad

This is my first time at GIDS. Amazing to know that this is the 7th such event and I didn't know about it till now.
Well as they say better late than never.

Coming to the highlights of the sessions here.

Why functional programming.. well great session by Venkat..

Things that Intel is doing with its RealSense platform. It is a great way to finding out how we will interact with the PC in the future.

One of the key sessions for me was the IOT by Janaki.
Also how the cloud is transforming hoe people use technology and how the landscape is changing for developers too.

The scale that these new technologies are designed to handle.

MEAN.JS session provides details the changing landscape of developing web applications.

Overall a great experience. Looking forward to it next year in a much bigger format at Hyderabad. Also with focus on Analytics and Big Data.

Big Data

Big Data

Over the last few weeks i have been trying to wrap my hands and head around this this Buzz word. Although, i do see a lot of material - blogs, articles, seminars around this new "kid" in the town i still haven't been able to understand it completely.

What is Big Data?
Primarily Big data means "BIG" Data. Over the last few years, mostly with the advent of web2.0, we have seen a prolific growth of social interactions over the web. This has led to a huge growth of data that is being shared over the internet.
Here i would like to go back to one of the questions that i was asked when i was in college:
What is the difference between "Data" and "Information"?
Data is any piece of fact that i available to us. however meaningful data is information.
Herein lies the answer to the question at hand. Companies over a period of time have accumulated a lot of data in terms of web logs, click stream data, server logs, etc. Until recently this data was simply discarded or just backed up in tape drives only to be thrown away. However, this data that the companies had was like a hidden treasure in order to conduct their business effectively. Click Stream data, could help companies understand customer behavior, which could lead for eg. e-commerce companies to potentially sell their products to them when they actually are looking for it.

So, now let us try to answer the original question:

1. Big data is any data from which you can derive meaningful information.
2. Big data is data that is normally unstructured - weblogs, server logs, etc.

In this blog I will try to start with the basics around Big data and talk about various advances happening in this area.

Merging Geometry boundaries using Oracle spatial

Some times there is a requirement to merge two or more boundaries/borders into one, such as a custom region/territory. For eg. you may need to merge California, Oregon and Washington which are the states that comprise the Western US region. 

In  order to achieve this, I assume that you already have the state boundaries for California, Oregon and Washington. If not, you can get it easily on otn, that includes the standard boundaries. in the world_sample schema. Once you have this information you can use the Oracle Spatial function SDO_AGGR_UNION. Hence, you can combine several standard boundaries into a new region.

Below is an way how you would merge the states mentioned above into a new custom Western US region.

INSERT INTO custom_area 

SELECT 'Western US', SDO_AGGR_UNION(SDOAGGRTYPE(o.geometry, 0.5)) FROM wom_area WHERE iso_country_code = 'USA' and feature_type = 909996 and name in ('CALIFORNIA', 'OREGON', 'WASHINGTON');

Here, the SDOAGGRTYPE is an object that specifies the geometry column from the spatial table wom_area and the dimensional array.

Create the Geometry columns in Oracle Database

Many times we have the Geo location related information as latitude and longitude. In order to plot these on the maps using mapviewer in OBIEE we need to convert these into spatial data which is a option available with the oracle database. In this entry we will look at how to convert the lat/lon information into Oracle Spatial data.

1. Create the Geometry columns in Oracle Database

a. Based on the latitude/longitude information that is available create Geometry column

b. Insert Geometry based columns in metadata tables (USER_SDO_GEOM_METADATA)

c. Create a spatial index

 

Let’s look at each of these steps in detail, with the help of an example. As mentioned earlier, the assumption is that the data we get from the customer has information on latitude and longitude. If the data that the customer provides does not have latitude-longitude information, the flow will change slightly to use the spatial capabilities of the database to convert the address into latitude and longitude.

 Step 1: Create the Geometry Columns in the Database

 There are two columns latitude and longitude that need to be available to be able to use the spatial capabilities of the Oracle Database.

1. Alter the table to create the Geometry column. In the example below, we create a column called store_geo in the “store” table

alter table store add (store_geo sdo_geometry);

 

2. Populate the new column using the following query:

update store set store_geo = MDSYS.SDO_GEOMETRY(2001, 8307, MDSYS.SDO_POINT_TYPE (longitude, latitude,NULL),NULL,NULL);

commit;

3. Check if the column has been populated correctly:

4. Update the spatial metadata table with the details for the new geometry column created

INSERT INTO USER_SDO_GEOM_METADATA (TABLE_NAME, COLUMN_NAME, DIMINFO, SRID)

VALUES ('store', 'store_geo',

MDSYS.SDO_DIM_ARRAY

(MDSYS.SDO_DIM_ELEMENT('LONG', -180.0, 180.0, 0.005),

MDSYS.SDO_DIM_ELEMENT('LAT', -90.0, 90.0, 0.005)

),

8307);

COMMIT;

5. Create the spatial index on the geometry column

DROP INDEX store_idx;

CREATE INDEX store_idx ON store(store_geo);

INDEXTYPE IS mdsys.spatial_index;