Data Extraction from SAP | Nitor Infotech
Send me Nitor Infotech's Monthly Blog Newsletter!
×
nitor logo
  • Company
    • About
    • Leadership
    • Partnership
  • Resource Hub
  • Blog
  • Contact
nitor logo
Add more content here...
Artificial intelligence Big Data Blockchain and IoT
Business Intelligence Careers Cloud and DevOps
Digital Transformation Healthcare IT Manufacturing
Mobility Product Modernization Software Engineering
Thought Leadership
Aastha Sinha Abhijeet Shah Abhishek Suranglikar
Abhishek Tanwade Abhishek Tiwari Ajinkya Pathak
Amit Pawade Amol Jadhav Ankita Kulkarni
Antara Datta Anup Manekar Ashish Baldota
Chandra Gosetty Chandrakiran Parkar Deep Shikha Bhat
Dr. Girish Shinde Gaurav Mishra Gaurav Rathod
Gautam Patil Harish Singh Chauhan Harshali Chandgadkar
Kapil Joshi Madhavi Pawar Marappa Reddy
Milan Pansuriya Minal Doiphode Mohit Agarwal
Mohit Borse Nalini Vijayraghavan Neha Garg
Nikhil Kulkarni Omkar Ingawale Omkar Kulkarni
Pooja Dhule Pranit Gangurde Prashant Kamble
Prashant Kankokar Priya Patole Rahul Ganorkar
Ramireddy Manohar Ravi Agrawal Robin Pandita
Rohan Chavan Rohini Wwagh Sachin Saini
Sadhana Sharma Sambid Pradhan Sandeep Mali
Sanjeev Fadnavis Saurabh Pimpalkar Sayanti Shrivastava
Shardul Gurjar Shravani Dhavale Shreyash Bhoyar
Shubham Kamble Shubham Muneshwar Shubham Navale
Shweta Chinchore Sidhant Naveria Souvik Adhikary
Sreenivasulu Reddy Sujay Hamane Tejbahadur Singh
Tushar Sangore Vasishtha Ingale Veena Metri
Vidisha Chirmulay Yogesh Kulkarni
Big Data | 29 Jun 2022 |   10 min

Data Extraction from SAP

featured image

In our technology-focused world, SAP ERP systems are very popular. Irrespective of the size or industry, businesses can reap the benefits of SAP. As you may be aware, it is essential to automate data extraction from SAP. If the idea sounds slightly daunting, it doesn’t need to be.

Remote Function Call (RFC) is the standard SAP interface for communication between SAP systems and other systems, so RFC calls function to be executed in a remote system. There are multiple ways to be developed to extract data from SAP tables. This blog covers the following two approaches to extract data from SAP ECC:

  • Python using the PyRFC package
  • Azure Data Factory with the SAP connector for .Net

I’m going to share the steps to use PyRFC and SAP NetWeaver SDK. Let’s get started!

Steps to download the SAP NetWeaver Remote Function Call System Development Kit

SAP NW RFC SDK is the abbreviation for SAP NetWeaver Remote Function Call System Development Kit. This lets you manage the RFCs.

1. Go to the SAP Service Marketplace: http://service.sap.com/support

2. Enter your SAP Service Marketplace username and password. The SAP Support Portal page appears. Here is the screenshot for your reference.

Data Extraction from SAP 1 | Nitor Infotech

Data Extraction from SAP 2 | Nitor Infotech

3. Click the ‘Download Software’ icon.

Data Extraction from SAP 3 | Nitor Infotech

4. Click the ‘Support Packages & Patches’ tab.

Data Extraction from SAP 4 | Nitor Infotech

5. Click ‘By Category’ in the list.

Data Extraction from SAP 5 | Nitor Infotech

6. Click ‘Additional Components’ in the list.

Data Extraction from SAP 6 | Nitor Infotech

7. Click ‘SAP NW RFC SDK’ in the list.

Data Extraction from SAP 7 | Nitor Infotech

8. Click on the most recent version of SAP NW RFC SDK.

Data Extraction from SAP 8 | Nitor Infotech

9. Select the operating system for which you want to download.

Data Extraction from SAP 9 | Nitor Infotech

10. Click the hyperlinked .zip file to download libraries.

Data Extraction from SAP 10 | Nitor Infotech

11. Extract the files and folders that are inside the .zip to your SAP folder.

Data Extraction from SAP 11 | Nitor Infotech

12. Install PyRFC Python package with the following commands –

pip install pyrfc

pip install pynwrfc

As we have done with pre-requisites, we can start extracting data from SAP tables using Python code.

Extract data from SAP tables and upload the data on Azure Blob Storage as CSV files using the following code snippet.

Import requires modules as mentioned below –

from azure.storage.blob import BlobServiceClient, BlobClient

from azure.storage.blob import ContentSettings, ContainerClient

from cryptography.fernet import Fernet

import pandas as pd

import json

from pyrfc import Connection

from datetime import datetime

import pyodbc

Read the SAP connection details from the configuration file kept in Blob Storage.

#Read connection details for configuration table

storage_end_point = "https://xxxxx.blob.core.windows.net"

storage_account_key = "xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx"

storage_container_name = "configurations"

blob_name = "config_connection.json"

key = "key.txt"

Establish the connectivity with Azure Blob Storage to read the configuration file.

blob_service_client_instance = BlobServiceClient(account_url= storage_end_point, credential =storage_account_key)

container = blob_service_client_instance.get_container_client(storage_container_name)

blobs = container.list_blobs(name_starts_with=blob_name)

blob = [i for i in blobs][0]

key_blobs = container.list_blobs(name_starts_with=key)

key = [i for i in key_blobs][0]

result = container.get_blob_client(blob).download_blob().readall()

result_json = json.loads(result.decode('utf8').replace("'",'"'))

Get SAP Server connection details from the configuration file.

print("-----Getting SAP Server details from configuration file------")

ashost = result_json['asHost']

sysnr = result_json['sysnr']

client = result_json['client']

user = result_json['user']

Decrypt the password to connect SAP.

print("------Decrypting SAP PWD------")

pwd = str.encode(result_json['sapPWD'])

key_result = container.get_blob_client(key).download_blob().readall()

fernet = Fernet(key_result)

decpwd = fernet.decrypt(pwd).decode()

decdbpwd = fernet.decrypt(dbpwd).decode()

Prepare connection string to connect the SAP server.

conn = Connection(ashost=ashost, sysnr=sysnr, client=client, user=user, passwd=decpwd)

print("------Connected SAP Successfully------")

Query SAP table using Remote function call RFC_READ_TABLE.

data = conn.call('RFC_READ_TABLE', QUERY_TABLE='KONV', DELIMITER='|', OPTIONS=options, FIELDS=['KNUMV', 'KPOSN', 'KSCHL', 'KWERT'], ROWSKIPS = rowskips, ROWCOUNT = ROWS_AT_A_TIME)

Write the data to Blob Storage in the form of a CSV file.

data_fields = data['DATA']

if not has_header:

data_names = data['FIELDS']

headers = [x['FIELDNAME'] for x in data_names]

for i in range (0, len(headers)):

column_name=headers[i]

if i == 0:

header_str = column_name

else:

header_str = header_str+"|"+column_name

csv_str = header_str

has_header = True

for row in data_fields:

csv_str=csv_str+'\n'+row['WA']

blob_client = blob_service_client_instance.get_blob_client(container='sapdata', blob='KONV.csv')

blob_client.upload_blob(csv_str, overwrite=True)

Data Extraction from SAP using Azure Data Factory and SAP connector for .Net

1. Download SAP connector for .Net from SAP website and install it on the virtual machine which can access the SAP server. This is required to establish the connectivity between SAP and Azure Data Factory through integration runtime.

2. Create a Linked Service in Azure Data Factory with all the required connection details as shown below.

Data Extraction from SAP 12 | Nitor Infotech

Data Extraction from SAP 13 | Nitor Infotech

3. Copy activity to copy the data from the SAP table in Azure Data Factory.

Data Extraction from SAP 14 | Nitor Infotech

Preview the result of any of the SAP tables before copying to any of the destinations.

Data Extraction from SAP 15 | Nitor Infotech

Shown below is the screenshot for the sample data from the SAP table.

Data Extraction from SAP 16 | Nitor Infotech

There you have it! These are the ways to extract data from SAP using NetWeaver SDK + Python and the SAP connector for .Net + Azure Data factory.

Send us an email with your thoughts about this blog and visit us at Nitor Infotech to understand how we partner with businesses to enhance their journeys of digital transformation. Also click here to discover what we offer in the realm of big data technology.

Related Topics

Artificial intelligence

Big Data

Blockchain and IoT

Business Intelligence

Careers

Cloud and DevOps

Digital Transformation

Healthcare IT

Manufacturing

Mobility

Product Modernization

Software Engineering

Thought Leadership

<< Previous Blog fav Next Blog >>
author image

Prashant Kamble

Lead Engineer

Prashant Kamble has more than 4.6 years of experience in the IT industry as a Lead Engineer. He has hands-on experience and knowledge in programming languages i.e. Python and ETL tools like Azure Data Factory and SSIS, business reporting tools like Power BI and Tableau, as well as database tools like MS-SQL server and SAP. Additionally, he has a good understanding of data warehouse and data modelling concepts.

   

You may also like

featured image

10 Heuristic Principles in UX Engineering

Say, you’ve built a modern, cutting-edge application. It has a complex, multi-layered user interface (UI), that is the basis for some amazing features. Since you’re the one who has built the applic...
Read Blog


featured image

ETL Testing: A Detailed Guide

Just in case the term is new to you, ETL is defined from data warehousing and stands for Extract-Transform-Load. It covers the process of how the data is loaded from the multiple source system to t...
Read Blog


featured image

Getting Started with ArcGIS Online

GeoServer is an open-source server that facilitates the sharing, processing and editing of geospatial data. When we are dealing with a large set of geospatial d...
Read Blog


subscribe

Subscribe to our fortnightly newsletter!

We'll keep you in the loop with everything that's trending in the tech world.

Services

    Modern Software Engineering


  • Idea to MVP
  • Quality Engineering
  • Product Engineering
  • Product Modernization
  • Reliability Engineering
  • Product Maintenance

    Enterprise Solution Engineering


  • Idea to MVP
  • Strategy & Consulting
  • Enterprise Architecture & Digital Platforms
  • Solution Engineering
  • Enterprise Cognition Engineering

    Digital Experience Engineering


  • UX Engineering
  • Content Engineering
  • Peer Product Management
  • RaaS
  • Mobility Engineering

    Technology Engineering


  • Cloud Engineering
  • Cognitive Engineering
  • Blockchain Engineering
  • Data Engineering
  • IoT Engineering

    Industries


  • Healthcare
  • Retail
  • Manufacturing
  • BFSI
  • Supply Chain

    Company


  • About
  • Leadership
  • Partnership
  • Contact Us

    Resource Hub


  • White papers
  • Brochures
  • Case studies
  • Datasheet

    Explore More


  • Blog
  • Career
  • Events
  • Press Releases
  • QnA

About


With more than 16 years of experience in handling multiple technology projects across industries, Nitor Infotech has gained strong expertise in areas of technology consulting, solutioning, and product engineering. With a team of 700+ technology experts, we help leading ISVs and Enterprises with modern-day products and top-notch services through our tech-driven approach. Digitization being our key strategy, we digitally assess their operational capabilities in order to achieve our customer's end- goals.

Get in Touch


  • +1 (224) 265-7110
  • marketing@nitorinfotech.com

We are Social 24/7


© 2023 Nitor Infotech All rights reserved

  • Terms of Usage
  • Privacy Policy
  • Cookie Policy
We use cookies to ensure that we give you the best experience on our website. If you continue to use this site we will assume that you are happy with it. Accept Cookie policy