Implementing an Open Data Platform using DKan on AWS

Common Open Data Platforms

In the Open Data concept, certain organizational data are freely available for anyone to use and republish as they wish, without any restrictions (copyrights, patents, etc.). This is similar to concepts such as Open Source, Open Hardware, Open Government, and Open Knowledge. It became more popular with the launch of Open Government Data (“Open Data”) initiatives adopted by world super powers (USA – data.gov, UK – data.gov.uk, India – data.gov.in). Most of these Open Data implementations were based on popular Open Data platforms/ frameworks already available in the market.

Some such Open Data Platforms are Ckan [1], Dkan [2], Socrata [3] and Junar [4]. Out of these, Ckan and Dkan are known as Open Source Open Data Frameworks, which are extremely popular in the Open Data market. Both Ckan and Dkan follow the “All-in-One” (centralized) and federated data management approaches, which were written in Python and Drupal/PHP languages respectively. Of these frameworks, Ckan and Dkan are equally popular.

About Dkan

Dkan is a Drupal-based community-driven Open Source Open Data platform offering a full suite of cataloging, publishing and visualization features. It allows organizations and individuals to have ultimate freedom to publish and open data to the public.

Why this blog?

Having its roots in Drupal, the official Dkan installation guide basically follows Drush [5] as the base installation. Hence, due to the unavailability of the relevant literature related to cloud deployment, having a complete and concise guide, in the form of this blog, would be beneficial to many Dkan cloud developers, I hope.

In the next few sections, I will walk you through the steps relatto setting up Dkan without downloading Drupal or its supporting modules.

Prerequisites

  • Pre-configured EC2 Ubuntu Instance (18.04) – (t2.small would suffice)
  • Basic knowledge about Ubuntu terminal

Steps

SSH to your EC2 instance and update the instance.

  • sudo apt-get update

After the update, install the following software to get it going.

01) Install Apache Server
  • Run the following command to Install Apache server  
  • - sudo apt-get install apache2
  • Verify the Apache server installation
  • - sudo systemctl status apache2
  • To stop, start and restart Apache server you can use following commands.
  • - sudo systemctl stop apache2
  • - sudo systemctl start apache2
  • - sudo systemctl restart apache2

02) Install Mysql Server


  • Run the following command to install Mysql server
  • - sudo service mysql stop

Once the installation is done, you can login to Mysql and create a database. To create a password for Mysql, we need to reset the root password and set a new one for the “root” user.

  • Let's start by stopping the running MySQL database:
  • - sudo service mysql stop
  • Next, create a /var/run/mysqld directory to be used by MySQL process to store and access the socket file:
  • - sudo mkdir -p /var/run/mysqld
  • - sudo chown mysql:mysql /var/run/mysqld
  • Once it is ready, manually start MySQL with the following command and options:
  • - sudo /usr/sbin/mysqld --skip-grant-tables --skip-networking &
  • To confirm that the process is running as expected, run the below command:
  • - Jobs
  • At this stage, it will allow us to access the MySQL database without a password:
  • - mysql -u root
  • Now flush privileges:
  • - FLUSH PRIVILEGES;
  • Next, we will reset root password as ”dkanpassword”:
  • - USE mysql;
  • - UPDATE user SET authentication_string=PASSWORD("dkanpassword") WHERE User='root';
  • - UPDATE user SET plugin="mysql_native_password" WHERE User='root';
  • Quit MySQL session:
  • - quit
  • Then terminate the current mysqld process:
  • - sudo pkill mysqld
  • Finally, start the MYSQL database:
  • - sudo service mysql start
  • If all went well, you should now be able to login to your MySQL database with the “root” password:
  • - mysql -u root --password -p
  • When it asks for the password provide it as “dkanpassword” and you will be able to login successfully. Then create a database with password. In this example I’m using “dkandb” as our database name.
  • - create database dkandb;
  • Then run the following command to verify that the database was created successfully.
  • - show databases;

03) PHP Installation with Required Modules

  • We’ve planned to install the PHP 7.0 version since it is deemed the most suitable PHP version for the Dkan integration.
  • - sudo apt install php7.0
  • In Dkan, there are some functions like image streams which are directed to a browser. You will need to compile PHP with the GD library for this to work.
  • - sudo apt install php7.0-gd
  • The DOM extension allows you to operate on XML documents through the DOM API with PHP.
  • - sudo apt-get install php7.0-dom
  • Php-mysql provides the mysql module for php.
  • - sudo apt-get install php7.0-mysql
  • In Drupal, we need to have the unicode support with a php mbstring extension for it.
  • - sudo apt-get install php7.0-mbstring
  • Create a simple PHP Info page to test your PHP installation., The following commands will create a new phpinfo.php file.
  • - sudo bash -c "echo -e '' > /var/www/html/phpinfo.php

Now type the URL “YOUR_PUBLIC_DNS/phpinfo.php” in your browser and you will see a page like this (See Figure 1).

04) Git Installation


  • Install Git using the following command.
  • - sudo apt-get install git

If the entire pre- installation is completed successfully, we are in a position to clone the Dkan source to the EC2 instance and proceed with the Dkan installation.

05) Dkan Installation


  • Before cloning the project, navigate to your webroot var/www/ html.
  • - cd /var/www/html/
  • Then clone the Dkan project source from Github.
  • - sudo git clone https://github.com/GetDKAN/dkan-drops-7.git
  • Next, we need to create and change certain file permissions in the Drupal codebase.
  • - sudo chmod go-w sites/default
  • - sudo mkdir sites/default/files
  • - sudo chmod a+w sites/default/files
  • - sudo cp sites/default/default.settings.php sites/default/settings.php
  • - sudo chmod 777 sites/default/settings.php

Thereafter, you can go to your browser and start the Dkan installation. Follow the below screenshot (See Figure 2) and continue the Dkan installation.

During the process, we are required to provide the database configuration as follows (See Figure 3). Please note, you have to provide your EC2 instance hostname as the database hostname.

The database installation will continue.

While the installation continues, it will display a progress bar on the screen (See Figure 4). Depending on your server resources, the complete process may take a few minutes.

        When the installation is complete, the site configuration screen will be displayed. Follow the prompts to set up your administrative username, email address, site name, time zone, and other default settings. If the final configuration is completed without any errors, you’ll see a short congratulatory message and you’ll be prompted to access your new site (See Figure 5).

On your web browser, you will see the Dkan admin panel as below (See Figure 6).

Congratulations! You have now successfully deployed Dkan on an AWS EC2 instance.

References

1. Ckan Web Site - https://ckan.org/
2. Dkan Web Site - https://getdkan.org/
3. Socrata - https://socrata.com/
4. Junar - http://www.junar.com/
5. Drush - A command line shell for Drupal - https://www.drush.org/ 

Supun Bandara

Associate Tech Lead

LET'S GET IN TOUCH

CONTACT US


PHONE
+1 919-943-6974 (USA)
+94112 337 516 (SL)
ADDRESS
130E, San Fernando Street,
#514 San Jose, CA 95112, USA

LOCATIONS

OUR GLOBAL PRESENCE