GIS Day is an international celebration of geographic information systems (GIS) technology. The event provides an opportunity for users of geospatial data and tools to build knowledge, share their work, and explore the benefits of GIS in their communities. Since its establishment in 1999, GIS Day events have been organized by nonprofit organizations, universities, schools, public libraries, and government agencies at all levels.
Held annually on the third Wednesday of November, this year GIS Day is officially today. Happy GIS Day! CDVS has participated in Duke GIS Day activities on campus in past years, but with COVID-19, we had to find other ways to celebrate.
A (Virtual) Map Showcase
To mark GIS Day this year, CDVS is launching an ArcGIS StoryMaps showcase! We invite any students, faculty, and staff to submit a story map to highlight their mapping and GIS work. Send us an email at firstname.lastname@example.org if you would like to add yours to the collection. We are keen to showcase the variety of GIS projects happening across Duke, and we will add contributions to the collection as we receive them. Our first entry is a story map created by Kerry Rork as part of a project for undergraduate students that used digital mapping to study the English CivilWars.
Why Story Maps?
If you aren’t familiar with ArcGIS StoryMaps, this easy-to-use web application integrates maps with narrative text, images, and video. The platform’s compelling, interactive format can be an effective communication tool for any project with a geographic component. We have seen a surge of interest in story maps at Duke, with groups using them to present research, give tours, provide instruction. Check out the learning resources to get started, or contact us at email@example.com to schedule a consultation with one of our GIS specialists.
One of the attractive features of Tableau for visualization is that it can produce maps in addition to standard charts and graphs. While Tableau is far from being a full-fledged GIS application, it continues to expand its mapping capabilities, making it a useful option to show where something is located or to show how indicators are spatially distributed.
Here, we’re going to go over a few of the Tableau’s mapping capabilities. We’ve recorded a workshop with examples relating to this blog post’s discussion:
Tableau is a visualization tool: Tableau can quickly and effectively visualize your data, but it will not do specialized statistical or spatial analysis.
Tableau makes it easy to import data: A big advantage of Tableau is the simplicity of tasks such as changing variable definitions between numeric, string, and date, or filtering out unneeded columns. You can easily do this at the time you connect to the data (“connect” is Tableau’s term for importing data into the program).
Tableau is quite limited for displaying multiple data layers: Tableau wants to display one layer, so you need to use join techniques to connect multiple tables or layers together. You can join data tables based on common attribute values, but to overlay two geographic layers (stack them), you must spatially join one layer to one other layer based on their common location.
Tableau uses a concept that it calls a “dual-axis” map to allow two indicators to display on the same map or to overlay two spatial layers. If, however, you do need to overlay a lot of data on the same map, consider using proper GIS software.
Displaying paths on a map requires a special data structure: In order for tabular data with coordinate values (latitude/longitude) to display as lines on a map, you need to include a field that indicates drawing order. Tableau constructs the lines like connect-the-dots, each row of data being a dot, and the drawing order indicating how the dots are connected.
You might use this, for instance, with hurricane tracking data, each row representing measurements and location collected sequentially at different times. The illustration above shows Paris metro lines with the station symbol diameter indicating passenger volume. See how to do this in Tableau’s tutorial.
You can take advantage of Tableau’s built-in geographies: Tableau has many built-in geographies (e.g., counties, states, countries), making it easy to plot tabular data that has an attribute with values for these geographic locations, even if you don’t have latitude/longitude coordinates or geographic files — Tableau will look up the places for you! (It won’t, however, look up addresses.)
Tableau also has several built-in base maps available for your background.
Tableau uses the “Web Mercator” projection: This is the same as Google Earth/Maps. Small-scale maps (i.e., large area of coverage) may look stretched out in an unattractive way since it greatly exaggerates the size of areas near the poles.
Useful Mapping Capabilities
Plot points: Tableau works really well for plotting coordinate data (Longitude (X) and Latitude (Y) values) as points. The coordinates must have values in decimal degrees with negative longitudes being east of Greenwich and negative latitudes being south of the equator.
Time slider: If you move a categorical “Dimension” variable onto Tableau’s Pages Card, you can get a value-based slider to filter your data by that variable’s values (date, for instance, as in Google Earth). This is shown in the image above.
Heatmap of point distribution: You can choose Tableau’s “Density” option on its Marks card to create a heatmap, which may display the concentration of your data locations in a smoother manner.
Filter a map’s features: Tableau’s Filter card is akin to ArcGIS’s Definition Query, to allow you to look at just a subset of the features in a data table.
Shade polygons to reflect attribute values: Choropleth maps (polygons shaded to represent values of a variable) are easy to make in Tableau. Generally, you’ll have a field with values that match a built-in geography, like countries of the world or US counties. But you can also connect to spatial files (e.g., Esri shapefiles or GeoJSON files), which is especially helpful if the geography isn’t built into Tableau (US Census Tracts are an example).
Display multiple indicators: Visualizing two variables on the same map is always problematic because the data patterns often get hidden in the confusion, but it is possible in Tableau. Use the “dual-axis” map concept mentioned above. An example might be pies for one categorical variable (with slices representing the categories) on top of choropleth polygons that visualize a continuous numeric variable.
Draw lines from tabular data: Tableau can display lines if your data is structured right, as discussed and illustrated previously, with a field for drawing order. You could also connect to a spatial line file, such as a shapefile or a GeoJSON file.
We’ve just given an overview of some of Tableau’s capabilities regarding spatial data. The developers are adding features in this area all the time, so stay tuned!
As data driven research has grown at Duke, Data and Visualization Services receives an increasing number of requests for partnerships, instruction, and consultations. These requests have deepened our relationships with researchers across campus such that we now regularly interact with researchers in all of Duke’s schools, disciplines, and interdepartmental initiatives.
In order to expand the Libraries commitment to partnering with researchers on data driven research at Duke, Duke University Libraries is elevating the Data and Visualization Services department to the Center for Data and Visualization Sciences (CDVS). The change is designed to enable the new Center to:
Expand partnerships for research and teaching
Augment the ability of the department to partner on grant, development, and funding opportunities
Develop new opportunities for research, teaching, and collections – especially in the areas of data science, data visualization, and GIS/mapping research
Recognize the breadth and demand for the Libraries expertise in data driven research support
Enhance the role of CDVS activities within Bostock Libraries’ Edge Research Commons
We believe that the new Center for Data and Visualization Sciences will enable us to partner with an increasingly large and diverse range of data research interests at Duke and beyond through funded projects and co-curricular initiatives at Duke. We look forward to working with you on your next data driven project!
The Data and Visualization Services (DVS) Department can help you locate and extract many types of data, including data about companies and industries. These may include data on firm location, aggregated data on the general business climate and conditions, or specific company financials. In addition to some freely available resources, Duke subscribes to a host of databases providing business data.
Directories of Business Locations
You may need to identify local outlets and single-location companies that sell a particular product or provide a particular service. You may also need information on small businesses (e.g., sole proprietorships) and private companies, not just publicly traded corporations or contact information for a company’s headquarters. A couple of good sources for such local data are the ReferenceUSA Businesses Database and SimplyAnalytics.
From these databases, you can extract lists of locations with geographic coordinates for plotting in GIS software, and SimplyAnalytics also lets you download data already formatted as GIS layers. Researchers often use this data when needing to associate business locations with the demographics and socio-economic characteristics of neighborhoods (e.g., is there a lack of full-service grocery stores in poor neighborhoods?).
Government surveys ask questions of businesses or samples of businesses. The data is aggregated by industry, location, size of company, and other criteria and typically include information on the characteristics of each industry, such as employment, wages, and productivity.
Macroeconomic indicators relate to the overall business climate, and a good source for macro data is Global Financial Data. Its data series includes many stock exchange and bond indexes from around the world.
Private firms also collect market research data through sample surveys. These are often from a consumer perspective, for instance to help gauge demand for specific products and services. Be aware that the numbers for small geographies (e.g., Census Tracts or Block Groups) are typically imputed from small nationwide samples, based on correlations with demographic and socioeconomic indicators. Examples of resources with such data are SimplyAnalytics (with data from EASI and Simmons) and Statista (mostly national-level data).
You may be interested in comparing numbers between companies, ranking them based on certain indicators, or gathering time-series data on a company to follow changes over time. Always be aware of whether the company is a publicly traded corporation or is privately held, as the data sources and availability of information may vary.
Developing interactive maps that incorporate text, images, video, and audio can be time-consuming and require specialized technical skills. Fortunately, at Duke we have access to Esri Story Maps, a web-based tool that helps you quickly design engaging narratives around your maps, no coding required.
We have seen a variety of creative uses of Story Maps at Duke, including:
Presentations to communicate research
Student assignments, as an alternative to a midterm or paper
Tours and guides of campus
Tutorials to explain a topic with a spatial component
Portfolios to showcase projects that include maps
Christine Liu, a graduate student in the Department of Art, Art History & Visual Studies, created this Story Map to illustrate a journey through Kraków under Nazi occupation.
If you are interested in building a Story Map, we recommend first spending some time exploring Esri’s curated gallery of stories to find inspiration and understand the platform’s capabilities. You can also review their collection of resources, which includes training videos, FAQs, and useful advice.
Data and Visualization Services is happy to announce its Fall 2016 Workshop Series. Learn new ways of enhancing your research with a wide range of data driven research methods, data tools, and data sources.
Interested in getting started in data driven research or exploring a new approach to working with research data? Data and Visualization Services’ spring workshop series features a range of courses designed to showcase the latest data tools and methods. Begin working with data in our Basic Data Cleaning/Analysis or the new Structuring Humanities Data workshop. Explore data visualization in the Making Data Visual class. Our wide range of workshops offers a variety of approaches for the meeting the challenges of 21st century data driven research. Please join us!
Ever wonder what the difference between a shapefile and a geodatabase is in GIS and why each storage format is used for different purposes? It is important to decide which format to use before beginning your project so you do not have to convert many files midway through your project.
Basics About Shapefiles:
Shapefiles are simple storage formats that have been used in ArcMap since the 1990s when Esri created ArcView (the early version of ArcMap 10.3). Therefore, shapefiles have many limitations such as:
Takes up more storage space on your computer than a geodatabase
Do not support names in fields longer than 10 characters
Cannot store date and time in the same field
Do not support raster files
Do not store NULL values in a field; when a value is NULL, a shapefile will use 0 instead
Users are allowed to create points, lines, and polygons with a shapefile. One shapefile must have at least 3 files but most shapefiles have around 6 files. A shapefile must have:
.shp – this file stores the geometry of the feature
.shx – this file stores the index of the geometry
.dbf – this file stores the attribute information for the feature
All files for the shapefile must be stored in the same location with the same name or else the shapefile will not load. When a shapefile is opened in Windows Explorer it will look different than when opened in ArcCatalog.
Better performance capabilities than Personal Geodatabase
Many users can view data inside the File Geodatabase while the geodatabase is being edited by another user
The geodatabase can be compressed which helps reduce the geodatabases’ size on the disk
On the other hand, Personal Geodatabases were originally designed to be used in conjunction with Microsoft Access and the Geodatabase is stored as an Access file (.mdb). Therefore Personal Geodatabases can be opened directly in Microsoft Access, but the entire geodatabase can only have 2 GB of storage.
To organize your data into themes you can create Feature Datasets within a geodatabase. Feature datasets store Feature Classes (which are the equivalent to shapefiles) with the same coordinate system. Like shapefiles, users can create points, lines, and polygons with feature classes; feature classes also have the ability to create annotation, and dimension features.
In order to create advanced datasets (such as add a network dataset, a geometric network, a terrain dataset, a parcel fabric, or run topology on an existing layer) in ArcGIS, you will need to create a Feature Dataset.
You will not be able to access any files of a File geodatabase in Windows Explorer. When you do, the Durham_County geodatabase shown above will look like this:
When you copy shapefiles anytime, use ArcCatalog. If you use Windows Explorer and do not select all the files for a shapefile, the shapefile will be corrupt and will not load.
When using a geodatabase, use a File Geodatabase. There is more storage capacity, multiple users can view/read the database at the same time, and the file geodatabase runs tools and queries faster than a Personal Geodatabase.
Use a shapefile when you want to read the attribute table or when you have a one or two tools/processes you need to do. Long-term projects should be organized into a File Geodatabase and Feature Datasets.
Many files downloaded from the internet are shapefiles. To convert them into your geodatabase, right click the shapefile, click “Export,” and select “To Geodatabase (single).”
Data and Visualization Services is happy to announce its Fall 2015 Workshop Series. With a range of workshops covering basic data skills to data visualization, we have a wide range of courses for different interests and skill levels.. New (and redesigned) workshops include:
OpenRefine: Data Mining and Transformations, Text Normalization
Advanced Excel for Data Projects
Analysis with R
Webscraping and Gathering Data from Websites
Workshop descriptions and registration information are available at: