Wednesday, March 30, 2016

Small Things of Big (Geospatial) Data

Be it clicks or likes; the networked world today is generating vast amounts of data at a significantly increasing rate. This together with exponentially increasing internal data is resulting in data explosion, popularly called “Big Data”.

The history of Big Data traces its origin to 1940s; the earliest documented use of the term “information explosion” and today we have Big Data explosion in geography (geospatial data).

Big data is a broad term for data sets so large or complex that traditional data processing applications are inadequate (Wikipedia). Paleolithic petroglyphs to modern data centers, the human race has always dealt with information. With technology innovation, Moore’s law is becoming irrelevant and Parkinson's Law of Data; “Data expands to fill the space available for storage” is resulting in information overflow or so called Big Data.

The world of Big Data is unfolding dramatically right before us from the amount of data being generated to the way in which it is structured and used. Despite the ever growing interest on “Big Data”, we surprisingly hear little about “Big Geospatial Data.” Nevertheless geospatial data has always been “Big Data”. Thanks to the advancements in geospatial data collection or acquisition such as satellite, remote sensing, global navigation satellite systems, aerial surveys using photographic / digital cameras, sensor networks, LiDAR and now Internet of Things (IoTs), is leading to exponential growth in volume of geospatial data. Big (Geospatial) Data exceeding capacity of current computing systems, presents its own set of opportunities and challenges. Examples of Big (Geospatial) Data include, but not limited to earth observation data, sensor data, location information, spatio-temporal data; which are key inputs for real time monitoring and management class of geospatial applications.

Organisations that generate and or consume geospatial data, suddenly find themselves swimming in so much data that they don't know what to do with them, besides faced with challenges such as capture, query, analysis, visualization, and dissemination of this data.

Contrary to the notion that Big (Geospatial) Data is just about handling lot more data (both structured and un-structured) requiring sophisticated data storage systems, organisations embracing Big (Geospatial) Data fail to take note that, it is also about handling

 Volume: unprecedented growth in data volumes
 Velocity: rapid increase in the velocity or speed of data creation (generation)
 Variety: extension in the variety of data types to be handled

Big (Geospatial) Data is more specifically concerned with the advancing speed in which new information, from an increasing number of diverse data sources and types, confront organisations trying to embrace Big (Geospatial) Data.

As with any Big Data, the three V’s: Volume, Velocity and Variety are the critical challenges with Big (Geospatial) Data as well. With efficient handling of these three V’s or “Small Things of Big (Geospatial) Data”, organisations embracing Big (Geospatial) Data is sure to reap the full benefits.

With FME (Feature Manipulation Engine), the industry standard in spatial data transformation technology, from Safe Software Inc., Canada you are in “Safe” hands for a reliable handshake with the ever growing Big (Geospatial) Data universe.

Volume: With support for several Big Data applications, FME eases the process of migration of voluminous (Volume) legacy or existing data of different types and models from/ to Big Data services or solutions.

Velocity: There is so much of geospatial and non-geospatial data today available for use immediately or in real time as soon as it is collected. In order for such real time data to be of any use, it requires suitable systems in place, to handle the advancing speed in which new information floods the system, from an increasing number of diverse data sources and types. FME’s real-time processing functionality has the ability to immediately respond to events and trigger different FME actions to ensure real-time data is delivered exactly how it's needed in actual real-time.

Variety: FME’s exceptional data conversion, transformation, integration, validation and migration capabilities together with support for 345+ data sources such as BIM/ CAD/ Database/ GIS/ LiDAR/ Raster/ Vector/ Web/ XML/ Sensor file formats and data model barriers; can help organisations solve Big (Geospatial) Data challenges with ease.

With one powerful integration engine and three ways to deploy: FME Desktop, FME Server and FME Cloud, keep ahead of evolving Big (Geospatial) Data, while addressing the three critical V’s or “Small Things of Big (Geospatial) Data”. While FME Desktop lets you connect and transform data in limitless ways, FME Server provides enterprise-level access to FME's powerful capabilities. FME Cloud is the hosted version of FME Server with no hardware required.

Despite several challenges posed by Big (Geospatial) Data, it has the potential to improve operations and make quicker and intelligent decisions. Big (Geospatial) Data when converted, transformed, shared and or integrated appropriately using a platform like FME, can help organisations gain useful insight to increase revenues, better manage its assets and improve operations geospatially, not just in a BIG way but also in a “Safe” way!

Wednesday, March 23, 2016

Geospatial Intelligence: The “Safe” Way

The intelligence, defence and law enforcement communities increasingly rely on geospatial intelligence in today’s world of evolving terror threats, theatres of war, natural disasters and civil unrest.

The ability of GIS to add spatial perspective to any data being analysed or monitored is the key to geo-intelligence. “Data is Power” and a key component that drives any geo-intelligence solution. Thanks to sensors and IoT (Internet of Things), big data is now a reality: the volume, variety and velocity of data coming into any geo-intelligence system continue to reach unprecedented levels.

Geo-intelligence is about timely collection, exploitation and analysis of geospatial data, including sensor data, imagery etc to maximise operational capabilities, assess risk and informed decision-making. “Right Information” at the “Right Time” in the “Right Form”

Many a time, at the critical hour, when the need is for a common operational picture the intelligence agencies end up with GIS interoperability issues not just against GIS file formats but also against real time information from sensors and IoTs. GIS interoperability is the ability to integrate or exchange information between different components of any geospatial (intelligence) solution, even though different agencies are on different GIS platforms and therefore file formats and data models.

GIS interoperability has long been a burning issue in the geo-intelligence community. For years geo-intelligence users and decision makers continue to maintain data locked up in some proprietary format until the gloomy cloud of interoperability issue suddenly snuck out from nowhere and its dark shadow descends over when the nation is under threat or a disaster strikes. A nightmare scenario, but for this digital alchemy: FME from Safe Software Inc., Canada, the industry standard in spatial data transformation technology.

Geo-intelligence community is an always connected world and FME’s data conversion, transformation, integration, validation and migration capabilities can not only empower this community to overcome 325+ CAD/GIS/Database/Web/Sensor file formats and data model barriers but also solve day-to-day data interoperability challenges.

With FME Desktop, FME Server and FME Cloud, keep ahead of evolving sensors or IoT technology and the ever increasing quantity of disparate data types to integrate and instantly make use of urgent intelligence. Increase operational efficiency by removing data integration problems. Build situational awareness by collecting, filtering, and analyzing information from vast data volumes to effectively identify threats and share critical knowledge. Make sense of your sensor data with FME to derive useful, actionable information from sensors with FME's ability to read and process sensor messages, analyze them and identify significant events, then execute appropriate responses to these events, all automatically and in real or near real-time.

FME Server is a complete sensor data processing solution that enables users to productively overcome all of the complications associated with utilizing sensor data. FME technology supports data transformation and integration between JSON, XML and CSV- the most common formats for raw sensor messages - and hundreds of other data types and applications, enabling you to use sensor data effectively with other data and in any system you need. Accomplish complex analysis with sensor networks, alert people and systems of sensor events in real-time, connect to sensor data management platforms and maximize the value of sensor data with FME.

With sensors in your own pockets, no matter whatever be the sensor data or GIS file formats that you handle, the only time GIS interoperability could be fun, is when you make peace with GIS file formats and sensor data using FME!

Thursday, February 26, 2015



Writing to CartoDB using FME: My first CartoDB visualization

Writing to CartoDB using FME: My first CartoDB visualization

Availability of schooling facilities in habitations at primary level as on 30th September 2009 using data from data.gov.in


Inspired by the new Writer for CartoDB in FME 2015 from Safe Software (www.safe.com), here is my first CartoDB visualization based on data from data from data.gov.in portal of Government of India under NDSP on availability of schooling facilities in habitations at primary and upper primary stages as on 30th September 2009


Click here for more information on CartoDB support in FME

Monday, September 08, 2014

Mr M.Krishnan's-Thirumullaivoyal house in Google Earth 3D

Well the intention of this blog as it states is to talk about Geographical Information Systems (GIS) and my (Saint GIS) favourite Spatial ETL tool FME (Feature Manipulation Engine),but this is an exceptional post for my well wisher.
It is said, one of the biggest commitments anybody will make in one’s life, both financially and emotionally, is buying or building a house. Though it is interesting for a few, the experience of most of the buyers is fraught with frustration, exhaustion and poor satisfaction. A significant feature is that the location of a project plays a predominant role in the decision making process. This is a 3D model of a residential house being constructed and modeled in 3D in Google Earth kml format for visualizing the proposed residential development along with its environs. This is a regular practice in many countries to ensure that any new urban development is in harmony with its neighbourhood.
So why not consider this as a first model(step) for Chennai!




Tuesday, June 25, 2013

Himalayan Tsunami 2013-Uttarakhand Flood Calamity 2013,India


Google Earth tour of some flood ravaged areas


This Google Earth project is dedicated to those pilgrims and tourists whose lives were lost in the recent floods (June 2013) in the State of Uttarakhand, India


This is a tour of some flood ravaged areas.


More information on Uttarakhand:

http://en.wikipedia.org/wiki/Uttarakhand
http://www.uk.gov.in

RESCUE & RELIEF OPERATIONS WEBSITE

http://164.100.150.41/rescuenrelief

Google Person Finder-2013 Uttrakhand Floods

http://google.org/personfinder/2013-uttrakhand-floods/

Monday, September 05, 2011

Web GIS for Public Health

Heraclitus, a Greek philosopher known for his doctrine of change being central to the universe rightly said that change is the only constant in this universe. No reason why this change should not be constant in the noble life saving field of medical sciences.

Public health is "the science and art of preventing disease, prolonging life and promoting health through the organized efforts and informed choices of society, organizations, public and private, communities and individuals" (1920, C.E.A. Winslow). It is concerned with threats to the overall health of a community based on population health analysis. The population in question can be as small as a handful of people or as large as all the inhabitants of several continents (for instance, in the case of a pandemic). Public health is typically divided into epidemiology, biostatistics and health services. Environmental, social, behavioral, and occupational health are other important subfields (Wikipedia). The underlying commonality in many of the above aspects of public health research is the spatial component of the factors being analysed or monitored.

Medical geography aka health geography, is an area of medical research that incorporates geographic techniques into the study of health around the world and the spread of diseases. It also includes studies on the impact of climate and location on an individual's health as well as the distribution of health services. Medical geography is an important field because it aims to provide an understanding of health problems and improve the health of people worldwide based on the various geographic factors influencing them. Health geography is the application of geographical information, perspectives, and methods to the study of health, disease, and health care (Wikipedia).

Thanks to the Greek doctor Hippocrates (5th-4th centuries BCE) for having studied the effect of location on one’s health thus laying the foundation for the beginning of medical geography. But for Dr John Snow, a doctor in London who plotted the distribution of cholera deaths throughout London on a map and found a cluster of unusually high deaths near a water pump on Broad Street, when the cholera epidemic gripped London medical geography would not have gained significance. Since then geographic techniques have found their place in several other areas of public health research.

The ability of a GIS to add spatial perspective to any data being analysed or monitored is the key to its application in public health research. GIS has always proved to be very useful to epidemiologists across the globe in elucidating patterns and relationships between the person, place, and time components of epidemiologist data. In addition, GIS technology has been an important tool for understanding and displaying disease or disease risk that are related directly to environmental exposure.

Medical geography has an increasing number of applications due to advancements in ICT and GIS. Besides, the spatial distribution of disease is still a large matter of importance, with GIS based maps playing a significant role in this field. Much recent attention has focused on developing GIS functionality in the Internet, Worldwide Web, and private intranets and is termed Web GIS. Web GIS is a Geographic Information System distributed across a networked computer environment to integrate, disseminate, and communicate geographic information visually on the World Wide Web over the Internet” [Gillavry, 2000]. Web GIS is also popularly called as Internet GIS. Web GIS holds the potential to make distributed geographic information (DGI) available to a very large worldwide GIS audience. On similar lines, Google Maps in the recent years has revolutionized the way in which information on several epidemics/ pandemics like swine flu, flu (Google Flu Trends) is delivered to general public with the latest addition being Google Insights for Search.

The objective is to focus on the following aspects:

  • Importance of public health research,
  • Current trends in the field of GIS, especially Web GIS
  • GIS Applications in Medical Geography – Tools & Technologies,
  • Spatial Epidemiology: Current Trends and Future Challenges
It can be easily demonstrated how the convergence of advancements in ICT and Medical Geography or Health GIS could play a major role in public health research in taking it to the next level.

The Future of Human Life Expectancy: Have We Reached the Ceiling or is the Sky the Limit? Although human ability to take command of the course of life and death is controversial, after remaining fairly constant for most of human history, life expectancy has nearly doubled in the past century. However the trend towards longer life has also raised concerns about the quality of life at older ages.

GIS together with the advancements in ICT, if applied in the right manner, at the right time, for sure can help address the problems confronting the epidemiologist and the medical community in saving the human race, if not solve them completely.

No matter whether human life expectancy has reached the ceiling or sky is the limit: Let us explore ways to apply GIS to make our lives better!

~ SRG

Saturday, December 18, 2010

GIS based Donor Information System

GIS based Donor Information System

Phone calls to friends and relatives with a great deal of anxiety are quite common during medical emergencies arising out of accidents or illness or other such medical conditions. Any help from unknown quarters comes as a big relief. But one is often caught scrambling during such critical times for much-needed information, the information regarding donors.

It is needless to emphasize the plight of those looking out for blood donors especially of rare blood groups during such medical emergencies. If such is the situation with information on blood donors, imagine the nightmare of those looking for information on donors of other types such as bone marrow, kidney, liver, lungs, eyes etc not getting the required information at the appropriate time. Everyday thousands of lives are lost for want of such critical information at the appropriate time. When lives are at stake, every second counts and words cannot describe the plight of the near and dear ones, running from pillar to post in locating the donors.

Geography or GIS for that matter has never been so closer offering a helping hand in our day-today life. Web based and even GIS based blood bank/ blood donor information systems are already the order of the day. But for some reason the concept has not extended beyond blood banks and blood donors into other types of medical donors like eye, kidney, bone marrow, lung, liver donors etc. and corresponding donor banks/ facility.

“Brain death” something hitting the headlines these days refers to the irreversible end of all brain activity (including involuntary activity necessary to sustain life) due to total necrosis of the cerebral neurons following loss of blood flow and oxygenation. Brain stem death (not whole brain death) is taken to be the significant indicator of death. Brain death may result in legal death, but still with the heart beating, and with mechanical ventilation all other vital organs may be kept completely alive and functional, providing optimal opportunities for organ transplantation. Most organ donation for organ transplantation is done in the setting of brain death. In some nations (for instance, Belgium, Poland, Portugal and France) everyone is automatically an organ donor, although some jurisdictions (such as Singapore, France, or New Zealand) allow opting out of the system. Elsewhere, consent from family members or next-of-kin is required for organ donation. (Source: http://en.wikipedia.org/wiki/Brain_death).

With the success of organ transplantation as an effective modality of treating end stage disease of various organs, increasing numbers of organ transplants are being performed all over the world. However, this procedure requires a “donor” pool of either “living” or “cadaveric” donors. Since this pool is limited, the gap between “demand” and supply is widening, which is further hampered with the non-availability of information regarding this limited donor pool at the right time. In the context of organ donation “cadaveric” donation has largely meant “brain dead” or “heart beating” donors. Such cadaver organ donation no doubt requires a robust IT/GIS based operational support system as its backbone to effectively utilize such cadaver donors.

A final thought: For a moment let us close our eyes and imagine the plight of the near and dear ones scrambling for information on donors, during medical emergencies. How thoughtful it would be if a donor registered for eye donation breathes his last and a SMS with the donor ID to a centralised server with details regarding the place, date and time of death etc is dispatched by the kith and kin of the deceased. This SMS then triggers of a set of GIS analysis on the server based on parameters such as the list of recipients registered in the vicinity of the donor considering medical factors for organ donation along with the spatial separation between the donor, recipient and the transplantation facility. The end result is an alert to either to the recipient/ care taker/ medical facility with location details of the donor.

A new concept such as this GIS based donor information system definitely needs inputs and support from all quarters and needs to be experimented, given a deep thought to put in practice.

The donors may be living or cadaver, their hearts beating or not beating, but let our hearts beat as one for a GIS based Donor Information System!

Can GIS Save Lives?

Can GIS save lives?

Medical geography aka health geography, is an area of medical research that incorporates geographic techniques into the study of health around the world and the spread of diseases. It also includes studies on the impact of climate and location on an individual's health as well as the distribution of health services. Medical geography is an important field because it aims to provide an understanding of health problems and improve the health of people worldwide based on the various geographic factors influencing them. Health geography is the application of geographical information, perspectives, and methods to the study of health, disease, and health care (Wikipedia)

During the times of the Greek doctor Hippocrates (5th-4th centuries BCE), people have studied the effect of location on one’s health. For example, early medicine studied the differences in diseases experienced by people living at high versus low elevation. It was easily understood that those at living low elevations near waterways would be more prone to malaria than those at higher elevations or in drier, less humid areas (geography.about.com). Though the reasons for these variations were not fully understood at the time, the study of this spatial distribution of disease for sure was the beginning of medical geography. It was not until middle of 18th century when the cholera epidemic gripped London that medical geography gained significance.

During the last three decades, a powerful technology has quietly changed the way people view and live in their neighborhoods, towns, and cities. Most people remain unaware of GIS and its impact-an impact that is as far-ranging as it is useful-despite GIS having grown immensely in the last 15 years, despite hundreds of thousands of people now using the technology, and despite it affecting the daily lives of millions (ESRI).

Be it the pandemic killer swine flu or AIDS of the 20th century, the super power of GIS is its ability to add a spatial perspective to any data being analysed. This has always proved to be very useful to epidemiologists across the globe in elucidating patterns and relationships between the person, place, and time components of epidemiologist data. In addition, GIS technology has been an important tool for understanding and displaying disease or disease risk that are related directly to environmental exposure.

Today, medical geography has a number of applications as well. Since the spatial distribution of disease is still a large matter of importance, with GIS based maps playing a significant role in this field. Google Maps in the recent years has revolutionized the way in which information on several epidemics/ pandemics like swine flu, flu (Google Flu Trends) is delivered to general public.

The Center for Disease Control and Prevention (CDC) in the United States for instance uses what they call the Atlas of United States Mortality to look at a wide range of health factors across the U.S. Data ranges from the spatial distribution of people at different ages to places with the best and worst air quality. The World Health Organization (WHO) features health data for the world with its Global Health Atlas. In India NATMO (National Atlas and Thematic Mapping Organisation) prepares Health and Diseases Atlas for India besides other organisations.
Though GIS has been playing a significant role in the field of medical geography, there are quite a few stumbling blocks/ limitations as well. They could be anything from accurately tracking the first incidence of a disease, the accuracy of data collected to confidentiality laws that can complicate the reporting of a disease.

A final thought: For a moment let us close our eyes and imagine the plight of our human race threatened with a plethora of heath related issues in this 21st century. Of what use could be any technology if it cannot reach and benefit the common public suffering in the veritable concrete jungles, the so-called cities and the remote villages of India? It is a pity that technologies such as GIS and many such applications do not see the light of the day, for one reason or the other. Let us explore ways to apply GIS to make our lives better!

Now, Can GIS save lives? The answer is: on the lighter vein, GIS has been my bread winner for several years now. GIS may not be the paragon of all virtues but it could be important as one of several measures for addressing the problems confronting the epidemiologist and the medical community in saving the human race, if applied in the right manner, at the right time.