Categories
Nasscom Community

GST: Dynamic QR Code: Request for issuance of clarification and deferral of timelines for smooth implementation

Rule 46 of Central Goods and Services Tax Rules, 2017 (CGST Rules), vide sixth proviso, empowers the Government to notify requirement of Quick Response (QR) code on the invoice.

Further, Notification no 72/2019-Central Tax dated December 13 2019 was issued by Central Board of Indirect Taxes and Customs (CBIC) which required a registered person with aggregate turnover exceeding INR 500 crores in a financial year, to mandatorily have a QR code on the invoice issued to an unregistered person (B2C invoice). Vide Notification 14/2020-Central Tax dated March 21, 2020 and Notification No 71/2020-Central Tax dated September 30, 2020, the above notification was superseded and date for issuance of QR Code on B2C invoice was deferred to October 1, 2020 and December 1, 2020 respectively.

On February 23, 2021, CBIC issued Circular no 146/02/2021-GST clarifying that dynamic QR Code should contain the following information:
➢ Supplier GSTIN number
➢ Supplier UPI ID
➢ Payee’s Bank A/C number and IFSC
➢ Invoice number & invoice date,
➢ Total Invoice Value and
➢ GST amount along with breakup i.e. CGST, SGST, IGST, CESS, etc.

While these much-awaited clarifications are welcomed by the industry, there are still various aspects in relation to generation of QR code which require further clarification. These include clarity on definition of “unregistered persons”, treatment of sales made through e-commerce platforms or online applications, whether banks and other RBI approved payment service providers will be making available this solution to registered persons or will the QR code be self-generated (by the respective registered persons), amongst others.

Drawing a parallel from e-invoicing which was implemented from October 1, 2020, there was a formal outreach conducted with stakeholders and clear guidelines were made available on the manner of generation of e-invoice, data points required to generate e-invoice as well as on Invoice Reference Number (IRN) generation.

However, the aforementioned Circular clarifies constituents of QR code without providing much clarity on its generation. QR Code is a business reform and not just a tax law change. It will involve significant changes to be made to ERP systems/ Point of Sales/ Cash Register System/ connected payment solution systems, which will require time for end-to-end implementation.

For a smooth and successful implementation of a critical business change of this magnitude, it is imperative that taxpayers are given adequate time to configure and test their IT systems from a date post where there is complete clarity regarding the process flow.

In this regard, we have made a detailed representation to GST Policy Wing requesting them to issue appropriate clarifications to the issues so that Industry can work towards smooth implementation of QR code requirement. Further, we have also request that a time frame of six months be given to businesses to enable them to plan and effect changes for smooth implementation of QR code requirements. For this, we have requested for  extension in timelines for waiver of penalty up to 30 September 2021 (i.e. additional six months) by way of amendment in notification no 89/2020-Central Tax, dated November 29, 2020.

We will keep you posted on further developments in this regard.

The post GST: Dynamic QR Code: Request for issuance of clarification and deferral of timelines for smooth implementation appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Categories
Nasscom Community

Batch Normalization: A different perspective from Quantized Inference Model

 

 

Abstract 

The benefits of Batch Normalization in training are well known for the reduction of internal covariate shift and hence optimizing the training to converge faster. This article tries to bring in a different perspective, where the quantization loss is recovered with the help of Batch Normalization layer, thus retaining the accuracy of the model. The article also gives a simplified implementation of Batch Normalization to reduce the load on edge devices which generally will have constraints on computation of neural network models.

 

Batch Normalization Theory

During the training of neural network, we have to ensure that the network learns faster. One of the ways to make it faster is by normalizing the inputs to network, along with normalization of intermittent layers of the network. This intermediate layer normalization is what is called Batch Normalization. The Advantage of Batch norm is also that it helps in minimizing internal covariate shift, as described in this paper.

The frameworks like TensorFlow, Keras and Caffe have got the same representation with different symbols attached to it. In general, the Batch Normalization can be described by following math:

 

 

                                                                                                                                            Batch Normalization equation

 

Here the equation (1.1) is a representation of Keras/TensorFlow. whereas equation (1.2) is the representation used by Caffe framework. In this article, the equation (1.1) style is adopted for the continuation of the context.

 

Now let’s modify the equation (1.1) as below:

 

 

Now by observing the equation of (1.4), there remains an option for optimization in reducing number of multiplications and additions. The bias comb (read it as combined bias) factor can be offline calculated for each channel. Also the ratio of “gamma/sqrt(variance)” can be calculated offline and can be used while implementing the Batch norm equation. This equation can be used in Quantized inference model, to reduce the complexity.

Quantized Inference Model

The inference model to be deployed in edge devices, would generally integer arithmetic friendly CPUs, such as ARM Cortex-M/A series processors or FPGA devices. Now to make inference model friendly to the architecture of the edge devices, will create a simulation in Python. And then convert the inference model’s chain of inputs, weights, and outputs into fixed point format. In the fixed point format, Q for 8 bits is chosen to represent with integer.fractional format. This simulation model will help you to develop the inference model faster on the device and also will help you to evaluate the accuracy of the model.

e.g: Q2.6 represents 6 bits of fractional and 2 bits of an integer.

Now the way to represent the Q format for each layer is as follows:

  1. Take the Maximum and Minimum of inputs, outputs, and each layer/weights.
  2. Get the fractional bits required to represent the Dynamic range (by using Maximum/Minimum) is as below using Python function:

def get_fract_bits(tensor_float):
  # Assumption is that out of 8 bits, one bit is used as sign
  fract_dout = 7 - np.ceil(np.log2(abs(tensor_float).max()))

  fract_dout = fract_dout.astype('int8')

  return fract_dout

  1. Now the integer bits are 7-fractional_bits, as one bit is reserved for sign representation.
    4. To start with perform this on input and then followed by Layer 1, 2 …, so on.
    5. Do the quantization step for weights and then for the output assuming one example of input. The assumption is made that input is normalized so that we can generalize the Q format, otherwise, this may lead to some loss in data when non-normalized different input gets fed.
    6. This will set Q format for input, weights, and outputs.

Example:
Let’s consider Resnet-50 as a model to be quantized. Let’s use Keras inbuilt Resnet-50 trained with Imagenet.

#Creating the model

def model_create():

    model = tf.compat.v1.keras.applications.resnet50.ResNet50(

    include_top=True,

    weights='imagenet',

    input_tensor=None,

    input_shape=None,

pooling=None,

    classes=1000)

    return model

 

Let’s prepare input for resnet-50. The below image is taken from ImageNet dataset.

Elephant Image from Imgenet data 

 

def prepare_input():

      img = image.load_img(

            "D:\Elephant_water.jpg",

            target_size=(224,224)

            )

      x_test = image.img_to_array(img)

      x_test = np.expand_dims(x_test,axis=0)

      x = preprocess_input(x_test) # from tensorflow.compat.v1.keras.applications.resnet50 import preprocess_input, decode_predictions

      return x

Now Lets call the above two functions and find out the Q format for input.

      model = model_create()

      x     = prepare_input()

If you observe the input ‘x’, its dynamic range is between -123.68 to 131.32. This makes it hard for fitting in 8 bits, as we only have 7 bits to represent these numbers, considering one sign bit. Hence the Q Format for this input would become, Q8.0, where 7 bits are input numbers and 1 sign bit. Hence it clips the data between -128 to +127 (-2⁷ to 2⁷ -1). so we would be loosing some data in this input quantization conversion (most obvious being 131.32 is clipped to 127), whose loss can be seen by Signal to Quantize Noise Ratio , which will be described soon below.

If you follow the same method for each weight and outputs of the layers, we will have some Q format which we can fix to simulate the quantization.

       # Lets get first layer properties

       (padding, _) = model.layers[1].padding

       # lets get second layer properties

       wts = model.layers[2].get_weights()

       strides = model.layers[2].strides

       W=wts[0]

       b=wts[1]

 

       hparameters =dict(

                    pad=padding[0],

                    stride=strides[0]

                   )

         # Lets Quantize the weights .

         quant_bits = 8 # This will be our data path.

         wts_qn,wts_bits_fract = Quantize(W,quant_bits) # Both weights and biases will be quantized with wts_bits_fract.

         # lets quantize Bias also at wts_bits_fract

         b_qn = (np.round(b *(2<<wts_bits_fract))).astype('int8')

         names_model,names_pair = getnames_layers(model)

         layers_op = get_each_layers(model,x,names_model)

         quant_bits = 8

         print("Running conv2D")

         # Lets extract the first layer output from convolution block.

         Z_fl = layers_op[2] # This Number is first convolution.

         # Find out the maximum bits required for final convolved value.

         fract_dout = get_fract_bits(Z_fl)

         fractional_bits = [0,wts_bits_fract,fract_dout]

         # Quantized convolution here.
         Z, cache_conv = conv_forward(

                         x.astype('int8'),

                         wts_qn,

                         b_qn[np.newaxis,np.newaxis,np.newaxis,...],

                         hparameters,

                         fractional_bits)

Now if you observe the above snippet of code, the convolution operation will take input, weights, and output with its fractional bits defined.
i.e: fractional_bits=[0,7,-3]
where 1st element represents 0 bits for fractional representation of input (Q8.0)
2nd element represents 7 bits for fractional representation of weights (Q1.7).
3rd element represents -3 bits for the fractional representation of outputs (Q8.0, but need additional 3 bits for integer representation as the range is beyond 8-bit representation).

This will have to repeat for each layer to get the Q format.

Now the quantization familiarity is established, we can move to the impact of this quantization on SQNR and hence accuracy.

Signal to Quantization Noise Ratio

As we have reduced the dynamic range from floating point representation to fixed point representation by using Q format, we have discretized the values to nearest possible integer representation. This introduces the quantization noise, which can be quantified mathematically by Signal to Quantization noise ratio.(refer: https://en.wikipedia.org/wiki/Signal-to-quantization-noise_ratio)

As shown in the above equation, we will measure the ratio of signal power to noise power. This representation applied on log scale converts to dB (10log10SQNR). Here signal is floating point input which we are quantizing to nearest integer and noise is Quantization noise.
example: The elephant example of input has maximum value of 131.32, but we are representing this to nearest integer possible, which is 127. Hence it makes Quantization noise = 131.32–127 = 4.32.
So SQNR = 131.32² /4.32² = 924.04, which is 29.66 db, indicating that we have only attained close to 30dB as compared to 48dB (6*no_of_bits) possibility.

This reflection of SQNR on accuracy can be established for each individual network depending on structure. But indirectly we can say better the SQNR the higher is the accuracy.

Convolution in Quantized environments:

The convolution operation in CNN is well known, where we multiply the kernel with input and accumulate to get the results. In this process we have to remember that we are operating with 8 bits as inputs , hence the result of multiplication need at least 16 bits and then accumulating it in 32 bits accumulator, which would help to maintain the precision of the result. Then result is rounded or truncated to 8 bits to carry 8 bit width of data.

def conv_single_step_quantized(a_slice_prev, W, b,ip_fract,wt_fract,fract_dout):

 """

 Apply one filter defined by parameters W on a single slice (a_slice_prev) of the output     activation

 of the previous layer.

 Arguments:

 a_slice_prev -- slice of input data of shape (f, f, n_C_prev)

 W -- Weight parameters contained in a window - matrix of shape (f, f, n_C_prev)

 b -- Bias parameters contained in a window - matrix of shape (1, 1, 1)

Returns:

 Z -- a scalar value, result of convolving the sliding window (W, b) on a slice x of the     input data

"""

       # Element-wise product between a_slice and W. Do not add the bias yet.

       s = np.multiply(a_slice_prev.astype('int16'),W) # Let result be held in 16 bit

       # Sum over all entries of the volume s.

       Z = np.sum(s.astype('int32')) # Final result be stored in int32.

       # The Result of 32 bit is to be trucated to 8 bit to restore the data path.

       # Add bias b to Z. Cast b to a float() so that Z results in a scalar value.

       # Bring bias to 32 bits to add to Z.

       Z = Z + (b << ip_fract).astype('int32')

       # Lets find out how many integer bits are taken during addition.

       # You can do this by taking leading no of bits in C/Assembly/FPGA programming

       # Here lets simulate

       Z = Z >> (ip_fract+wt_fract - fract_dout)

       if(Z > 127):

             Z = 127

       elif(Z < -128):

             Z = -128

       else:

            Z = Z.astype('int8')

       return Z

The above code is inspired from AndrewNg’s deep learning specialization course, where convolution from scratch is taught. Then modified the same to fit for Quantization.

Batch Norm in Quantized environment

As shown in Equation 1.4, we have modified representation to reduce complexity and perform the Batch normalization.The code below shows the same implementation.

def calculate_bn(x,bn_param,Bn_fract_dout):

        x_ip = x[0]
        x_fract_bits = x[1]

       bn_param_gamma_s = bn_param[0][0]

       bn_param_fract_bits = bn_param[0][1]

       op = x_ip*bn_param_gamma_s.astype(np.int16) # x*gamma_s

       # This output will have x_fract_bits + bn_param_fract_bits

       fract_bits =x_fract_bits + bn_param_fract_bits

       bn_param_bias = bn_param[1][0]

       bn_param_fract_bits = bn_param[1][1]

       bias = bn_param_bias.astype(np.int16)

       # lets adjust bias to fract bits

       bias = bias << (fract_bits - bn_param_fract_bits)

       op = op + bias # + bias

       # Convert this op back to 8 bits, with Bn_fract_dout as fractional bits

       op = op >> (fract_bits - Bn_fract_dout)

       BN_op = op.astype(np.int8)

       return BN_op

Now with these pieces in place for the Quantization inference model, we can see now the Batch norm impact on quantization.

Results

The Resnet-50 trained with ImageNet is used for python simulation to quantize the inference model. From the above sections, we bind the pieces together to only analyze the first convolution followed by Batch Norm layer.

The convolution operation is the heaviest of the network in terms of complexity and also in maintaining accuracy of the model. So let’s look at the Convolution data after we quantized it to 8 bits. The below figure on the left-hand side represents the convolution output of 64 channels (or filters applied) output whose mean value is taken for comparison. The Blue color is float reference and the green color is Quantized implementation. The difference plot (Left-Hand side) gives an indication of how much variation exists between float and quantized one. The line drawn in that Difference figure is mean, whose value is around 4. which means we are getting on an average difference between float and Quantized values close to a value of 4.

                                                                                                                                    Convolution and Batch Norm Outputs

 

Now let’s look at Right-Hand side figure, which is Batch Normalization section. As you can see the Green and blue curves are so close by and their differences range is shrunk to less than 0.5 range. The Mean line is around 0.135, which used to be around 4 in the case of convolution. This indicates we are reducing our differences between float and quantized implementation from mean of 4 to 0.135 (almost close to 0).

 

Now let’s look at the SQNR plot to appreciate the Batch Norm impact.

                                                                                                                      Signal to Quantization Noise Ratio for sequence of layers

 

Just in case values are not visible, we have following SQNR numbers
Input SQNR : 25.58 dB (The Input going in to Model)
Convolution SQNR : -4.4dB (The output of 1st convolution )
Batch-Norm SQNR : 20.98 dB (The Batch Normalization output)

As you can see the input SQNR is about 25.58dB , which gets reduced to -4.4 dB indicating huge loss here, because of limitation in representation beyond 8 bits. But the Hope is not lost, as Batch normalization helps to recover back your SQNR to 20.98 dB bringing it close to input SQNR.

Conclusion

  1. Batch Normalization helps to correct the Mean, thus regularizing the quantization variation across the channels.
  2. Batch Normalization recovers the SQNR. As seen from above demonstration, we see a recovery of SQNR as compared to convolution layer.
  3. If the quantized inference model on edge is desirable, then consider including Batch Normalization as it acts as recovery of quantization loss and also helps in maintaining the accuracy, along with training benefits of faster convergence.
  4. Batch Normalization complexity can be reduced by using (1.4) so that many parameters can be computed offline to reduce load on the edge device.

References

 

The post Batch Normalization: A different perspective from Quantized Inference Model appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Categories
Nasscom Community

Top 7 reasons why it is important to backup your files in today’s age of cyberthreats

2017 was the year of biggest hacks, persistent and endless stream of breaches and leaks, that affected innumerable businesses and people across the globe.

  • Cybercriminals stole the personal data of over 148 million people by penetrating Equifax (EFX), a consumer credit reporting agency.
  • WannaCry, spanned more than 150 countries, leveraged some of the leaked NSA tools.
  • The malware NotPetya targeted Ukrainian businesses using compromised tax software.

This retrospective look at the state of cybersecurity in 2017 clearly indicates that 2017 was not a very safe year and importance of backup and recovery was strongly realized by many businesses. The time is now up for questions like why is data backup necessary or why should I backup data, rather it’s now time to know how to backup and which is the best enterprise backup solution for me.

Today, organizations of all sizes are relying more and more on bits and bytes, than on bricks and mortar for their business and this has thus increased data value significantly, and that in turn, requires increased levels of IT security.

Consequences of data loss for a business are dire – downtime, lost productivity, and long-term reputational damage.

As per a research*, 80 percent of businesses suffering a major disaster go out of business in three years, while 40 percent of businesses that experience a critical IT failure go out of business within one year. In the case of suffering a fire, 44 percent of enterprises fail to reopen and 33 percent of these failed to survive beyond 3 years.

Thus, organizations of all sizes need to protect their data, backup their files and computers and ensure that their systems can be restored smoothly, quickly and completely.

So, every IT manager should make sure that they have a Disaster recovery and backup plan in place to restore operations and avoid data loss.

What is backup, what is disaster recovery – is there a difference?

Backup and disaster recovery are two different terms, often confused as synonyms of each other.

Backups are copy of your files and data that can be used to bring a failed system back online, whereas disaster recovery includes set of tools and procedures that enable recovery of lost data and systems following man made or natural disasters.

But, backup has to be there in the first place, as without it, disaster recovery is not possible.

What should you backup?

Everything that cannot be replaced! Your data backup and recovery solution should cover everything from snapshots to streaming – all file types, databases, endpoints, applications, Virtual Machines, media storage and data sources.

As per TechTarget Research, a majority of businesses back up virtualized servers and databases. Others back up data sets that users share as well as enterprise applications, while few back up endpoint or desktop data and virtual desktops.

Top 7 reasons why it is important to backup your files in today’s age of cyberthreats 1

Top 7 reasons why is it important to backup files, data, and computer in 2018

1.Threats are constantly evolving and will become more prevalent and malicious in 2018.

Various reports, compiled by a number of prominent figures in the IT security sector, reveal that the number of cyber-attacks are growing year upon year, with as many as four times the amount occurring in 2017 than 2016.

Similarly, 2018 is also expected to witness plague of big risks, huge breaches, ransomware attacks and modified attacks with the technology advancements, like Artificial Intelligence (AI) powered hacking.

AI unfortunately gives attackers the tools to get a much greater return on their investment,” explains Steve Grobman, chief technology officer at McAfee.

As per MIT Technology review, “Companies that hold sensitive information will be in the sights of hackers in 2018.”

World Economic Forum’s Global Risk Report 2018 also ranks cyber-attacks as the third most likely global risk for 2018, after extreme weather conditions and natural disasters.

Prevention is the best form of protection and all organizations owe the responsibility to invest in well managed backup solutions which will empower them with the ability to prevent, detect and of course contain data breaches.

Thus, building resilience should be involved in all plans of business setup and execution.

2. Economic losses can be huge and serious

In today’s digital economy, data is the core currency of enterprises and downtime resulting from unplanned system disruption or data loss is expensive.

Below graph* shows the average hourly cost of critical server outages. As of December 2017, 24 % of respondents worldwide reported the average hourly downtime cost of their servers to be between 301,000 to 400,000 U.S. dollars.

Average cost per hour of enterprise server downtime worldwide in 2017 and 2018

Top 7 reasons why it is important to backup your files in today’s age of cyberthreats 2

So, the cost of downtime is expected to increase significantly for companies in 2018

Thus, a backup and recovery strategy is necessary to protect your mission critical data against planned or unplanned disruption and downtime originating from it.

3. Reputation and trust are at stake

Quantifying the consequences of data breached is a bit challenging as money is just one part of the equation, which can be calculated, but reputation and productivity loss cannot be measured.

“Consumers are starting to pull away from brands that have been breached,” says Christian A. Christiansen, IDC Program Vice President, Security Products and Services.

In addition to this, time is also lost as during recovery process, no new work can be initiated and this further delays your time to market and impacts ROI. In some cases, it may result in a legal challenge from consumers or compliance authorities.

And the reputational damage suffered by companies who fail to protect personal data can translate directly into a loss of business.

4. Big data is becoming more mission critical in nature

Forbes reported that Big data adoption reached 53% in 2017 up from 17 % in 2015.

This is an indication that big data is becoming a practical pursuit among organizations rather than an experimental endeavor. Across the three years of our comprehensive study of big data analytics, we see a significant increase in uptake in usage and a large drop of those with no plans to adopt,” said Howard Dresner, founder and chief research officer at Dresner Advisory Services.

Top 7 reasons why it is important to backup your files in today’s age of cyberthreats 3

Analytics tools offer both predictive and prescriptive insights, and this enables companies to store piles of data for as long as they need for getting better insights into their business and for taking more informed decisions while formulating future business strategies. But, this may also mean, massive amounts of both personal and business-critical information loss, if breached.

So as this “big data” grows in volume and is more complex and mission critical in nature, enterprises will need a big data disaster recovery and backup plan that can act as a protection layer against the effects of negative events like cyberattacks or equipment failures.

5. Digital Transformation is dominating many C- suite and boardroom agendas

Almost every organization is transitioning into a digital business some or the other way, but the first step to digital transformation is to manage risks – business must address their uptime and availability shortfalls. A digital business will need to establish a secure ecosystem that will help it survive threats originating from unplanned downtime caused by network outages, cyber-attacks and infrastructure failures.

As per the Gartner predictions by 2020, 60 percent of digital businesses will suffer major service failures due to the inability of IT security teams to manage digital risk.

Vice president and distinguished analyst at Gartner, Paul Proctor explains, “Cybersecurity is a critical part of digital business with its broader external ecosystem and new challenges in an open digital world.”

So, business units should learn to live with acceptable levels of digital risk and plan to protect their digital content with best backup and disaster recovery software.

6. Endpoints are increasing in businesses

Today more and more enterprise data is managed across different devices- laptops, smartphones, tablets etc. facilitating remote working culture and thus is increasing the number of endpoints that the data is stored on. These endpoints can be within your business’ computer network or outside it, sometimes they may be in public places or even at home.

In such cases, you need to maintain a backup protection for endpoint devices, so that even if these devices are lost, your data is safe in the central repository. Cloud-based data solutions are considered best as they provide geolocation and remote wipe capabilities to prevent data loss and ensure data security.

7. 2018 will see secondary data work smarter – data enablement

Your backed-up data can provide incremental value to your business, rather being only used as an insurance cover for negative outcomes. You can use it for data analysis and mining, patch testing, use non-production data for application testing and to ensure backward compatibility of new applications.

Most of the big backup companies are planning to use backed up data for different meaningful purposes in 2018 – “Now customers are starting to ask for it, so the bigger backup players are obliging,” says Jason Buffington, a data protection analyst with Enterprise Strategy Group (ESG). ESG refers to the concept as data management and enablement.

What is the best backup and DR strategy?

No doubt, a good backup strategy is the best defense against data loss. With inadequate data protection solutions, it becomes difficult to regain lost data and files when required.

Below screenshot from ESG Research depicts primary requirements of IT organizations for best backup and disaster recovery strategy- high recovery speed, high recovery reliability, high backup speed or frequency, and reduced hardware, software costs.

Top 7 reasons why it is important to backup your files in today’s age of cyberthreats 4

Considering this and the evolving requirements of resiliency and recoverability, IT organizations are adopting cloud backup solutions over traditional backup plans. They are seeking to extend their backup storage and data archives to the cloud for cost control, backup offsite vaulting, and archiving, achieving efficiency and meeting compliance requirements.

Top 7 reasons why it is important to backup your files in today’s age of cyberthreats 5

Cloud-based data backup is quickly gaining momentum as it offers improved and reliable backup, improved security and compliance and most importantly, reduced costs, in terms of resources. Also, it enables businesses to quickly restore normal operations, after a system failure or file loss.

Top 7 reasons why it is important to backup your files in today’s age of cyberthreats 6

Watch the two-minute video for quick overview-

One of the best online backup solutions that needs mention here is Azure cloud backup, the most cost effective and enterprise grade cloud storage service. With ityou can back up your data for, virtual machines, virtualized workloads, on-premises servers, SQL server, SharePoint server, and more.

Originally Published on ZNetLive

The post Top 7 reasons why it is important to backup your files in today’s age of cyberthreats appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Categories
Nasscom Community

A Swift Ladder to Cloud | How and What to Estimate during Cloud Migrations

The technology industry has been provided with options of various cloud platform solutions and the specific migration guides from the cloud platform owners. With decently structured recommendations,the business organizations can very well begin on their own to strategize the migration of their product work loads. How then is this paper different from already existing guidelines published by various cloud vendors? The answer to this curiosity is that this paper helps to provide a layer of practical insights and estimation areas of many implicitly required efforts,which have been collected and curated from the experiments on various small and large scale cloud migrations. The real time effort evidence comes not only from brownfield but greenfield executions that have been practically conducted to solve different core business challenges, different sizes of workloads, and different desired end-state architectures.This paper is intendedas a cheat book for architects, engineering program managers and lead developers who play a role of competent cloud partners to estimate and plan the efforts for enterprise migrations.

By Surbhi Nijhara

Senior Solution Architect, GlobalLogic India Ltd.

The post A Swift Ladder to Cloud | How and What to Estimate during Cloud Migrations appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Categories
Nasscom Community

MCA: Representation regarding onerous obligations prescribed under the Companies (CSR) Amendment Rules 2021

The Ministry of Corporate Affairs (MCA) recently notified Companies (Corporate Social Responsibility Policy) Amendment Rules, 2021 (Amendment Rules) to amend Companies (Corporate Social Responsibility Policy) Rules, 2014 (CSR Rules 2014). The Amendment Rules impose several new obligations on companies, which could result in hardship for companies.

In this regard, we have submitted a detailed representation to MCA on March 9, 2021 highlighting key issues faced by the Industry along with suggestions to address the same. Our suggestions to MCA include doing away with the requirement of certification by Chief Financial Officer or the person responsible for financial management, suggesting applicability of certain rules with prospective effect, allowing companies to carry forward excess CSR spending to next FY, amongst others.

The detailed suggestions  are attached for your reference.

Please write to tejasvi@nasscom.in in case you have any feedback.

 

The post MCA: Representation regarding onerous obligations prescribed under the Companies (CSR) Amendment Rules 2021 appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Categories
Nasscom Community

Leader Talk: In Conversation with Dr. Harsh Vinayak Senior Vice President, Intelligent Automation & Data Services, NTT DATA Services

   1. Share your thoughts on India as an emerging   hub of Innovation

India, for long, has been known as the land of “jugaad”. We have a penchant, because of perpetual scarcity, to devise innovative solutions. When we started the journey of IT Services a few decades ago, our business mostly comprised of staffing solutions, whose architects mostly resided at our clients in their onshore locations. We were contracted to follow these defined processes exactly. This necessitated strict process control, the famous “6 sigma” that we became so good at. While this was a necessity at that time, unfortunately it also discouraged our immense capability of “jugaad”!

However, today we have come a long way from the initial days of staff augmentation. For example, more than 65% of our business is now Outcome Based. This passes the baton of technology as well as process optimization to us as the Service Providers. The erstwhile maligned “jugaad” now becomes a necessity for continuously generating value for our clients, albeit it has now been rechristened and is cherished as “innovation” and “reimagining” and forms the bulwark of our offerings. In other words, it’s not that innovation is new to India, what has changed is the nature of IT Services business that now makes it not only feasible but an imperative to bring innovation to our clients. Not only is innovation becoming necessary to add value to our offerings, as the emerging hub of Global R&D, “Innovation” itself is fast becoming our Offering! My organization, NTT DATA is built on a culture of innovation, after all, we invest upwards of Four Billion Dollars every year on research and development and are the author of 1G all the way to 5G telecom services! We continue to explore new technologies through various global initiatives and India is now the focal point of our initiatives. In fact, the winner of this year’s NTT DATA Global Open Innovation Challenge is a start-up from India! In addition, similar to many other IT companies a large part of our Digital Innovation Engineering Centre is based out of India. I believe, India has formally arrived in the R&D Leadership circle.

  1. The role of Technology in driving the IT Services Landscape in India – how do you see it playing out?

IT Service, by its very definition is based on the foundation of emerging technology. However, as I mentioned above, a few decades ago, India emerged as the leader in this sector as a low-cost staffing services provider. While applauding their amazing service to our economy, it is evident that even the early Indian start-ups that are unicorns now, simply Indianized ideas that had been successful in the developed nations. Three decades ago, when I graduated from IIT Kanpur, 250 out of our class of 300, including myself, migrated to these nations that encouraged and rewarded innovative initiatives.

However, now the situation is very different! India’s offering is no more limited to low cost, low skilled labour; we are the hub of innovation, jugaad and R&D. We are pioneering new technologies that are leading Global IT Services. The newest offerings in AI/ML, Security, Healthcare, Financial Services, Insurance, Manufacturing and several other verticals are being transformed with fundamental innovative ideas originating in India by Indians – whether it be through Start-ups, Indian Tech. companies, GICs of multinationals, govt. and private Labs or universities.

Not only the R&D and Software Services Offerings, but even the Process Outsourcing Offerings are now being heavily influenced by digital technology platforms. There is hardly any significant IT/IT-enables Services deal that is merely based on labour – they are all multi-tower services where the differentiator is the innovative technology that the service provider brings to our clients. And once again the Indian arms of most of the IT Services companies are leading this IT-Services revolution; for example, more than two-thirds of NTT DATA Services Digital Platform Engineering team is based out of India.

  1. How is your organization supporting the youth – academic students and budding start-ups of India, and what are the plans?

NTT, with presence in over 80 countries, depends on local skilled employees and technology partnerships to serve our clients. To further this pursuit, we have numerous global and local initiatives that facilitate technology education of youths and the start-up community through technology-transfer, access to global markets and funding. In India, we have partnered with over fifty educational institutes to help develop their curriculum, make their graduates employable and ultimately provide them with hi-growth employment. Our recently concluded Open Innovation Challenge provided industry insights to promising young technology intensive companies which led to one of our Indian start-ups win the Global competition. This will open immense opportunity for these start-ups. Over the past ten years, we have developed close collaboration with key technology universities of India – IIT Delhi and now IIT Kanpur to guide and fund the innovators in their Start-up Incubators. This has resulted in main-streaming of key break-through technologies that are being employed to bring value to our clients. NTT DATA’s key mission “to partner with community to uplift society through technology innovation” runs through every engagement we have across the globe.

  1. What are the new technologies/models adapted by NTT DATA in the new normal. How does is contribute to the industry?

The pandemic has shown that the IT industry is in a strong position to help handle global challenges. In the last twelve months, we have developed strong expertise in understanding the trends and technologies impacting our clients and the overall business environment and with our strong R&D foundation have been leading the charge to address these opportunities in the new normal. This required cultural shift (people) and new capabilities (Technology) to transform from traditional outsourcing to innovation led Services model that impacts business outcomes. With our Enterprise Digital Platform, Nucleus© , already in an advanced stage of maturity, we were well on our way to bringing Intelligent Automation to our clients and help them tap new potentials through our Operative Insights. The pandemic hastened the adoption of these digital technologies. Now, we are leading every engagement with Automation first with our Deep Learning ANN models predicting the outcomes to an uncanny degree of accuracy. Our large Healthcare client base benefitted from our intelligent Contact Tracing Platform; while our AIOps based infrastructure automation made sure that the front-line workers and key government agencies, such as CDC, were ensured un-interrupted communications with the rest of the world. We enabled major State organizations as well as global fortune 500 companies to make the shift to remote work through our already mature comprehensive Dynamic Workforce Management Platform. As an example, when the City of New York needed to quickly pivot to on-line education, we partnered with Apple to seamlessly provision thousands of tablets overnight to underprivileged kids so that there was no break in their education.

The pandemic also changed the landscape of Customer Experience. Marketers across the globe extensively worked on their strategies, to connect well with our clients. Customer Experience has always played a pivotal role for NTT DATA and has been our key consulting service. And with an optimised MarTech we were able to deliver a redefined Digital Customer Experience. In this crisis, we felt digitally empowered like never before.

  1. Policy recommendations that you have specifically for the government

The Indian Government has been an ardent supporter of our industry, starting with the SEZ and STPI regulations that set-in motion the IT revolution that India leads today.  In short three decades this has given rise to a $200 Billion Indian IT Industry constitutes over 8% of India’s GDP and generates over 140,000 new jobs every year. However, a sector that had a 15% YoY growth is now growing at a more sedate 2.3%. One of the reasons for this is that “the world has become flatter”, to rephrase Thomas Friedman! No longer is the promise of low cost labour the primary driver of Indian IT Services business. The pandemic further emphasized the already growing realization that regional cost advantage is often trumped by technology led advantages. Hence to maintain India’s IT industry dominance, we need to be at the fore-front of the Digital tide, with technology innovators leading our way and Indian government policies paving a smooth path for this. Digital India Initiative, Make in India and “Atmnirbhar” policies are in the right directions – they lend support to the growing technology innovation base in India and places us in an advantageous position for future growth.

These show an eagerness and a willingness at the Centre to support the IT Industry, yet these do not often translate to ground level policies. The often-contentious State legislative structure suppresses the advantages that these central policies promise. For example, Central Government was quick to realize that the new post-pandemic Normal necessitates work-from-home as the predominant mode of service delivery and it made this intent very clear by formulating policies clearing the path. However, to this day, the local SEZ’s have not been notified on procedures to allow assets and employee registration, while working from home.

Another recent contentious issue that has become onerous to our industry are local employment laws that restrict free employment of deserving candidates regardless of their domicile status. Although NTT DATA encourages any policy that generates local employment, these ultra-restrictive laws, reminiscent of the Permit-Raj will only drive our productivity backwards and hinder us from unleashing the full potential of Make in India benefits to our global clients.

NASSCOM has been very diligent in representing the IT Industry to the Indian Government which has led to significant reforms furthering our contribution to the nation. More of it is needed.

The post Leader Talk: In Conversation with Dr. Harsh Vinayak Senior Vice President, Intelligent Automation & Data Services, NTT DATA Services appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Categories
Nasscom Community

Edge AI: Changing the Cyber Security Landscape through Scalable and Flexible Data Safety

Edge AI: Changing the Cyber Security Landscape through Scalable and Flexible Data Safety

The year 2020 was a watershed moment for technology adoption. The COVID-19 pandemic forced businesses around the world to transition to remote working and run operations through cloud-based platforms. Fields such as telehealth, e-commerce, and digital collaboration technologies also saw record growth. Experts anticipate that some of these changes might be here to stay. In fact, research reveals that in the US alone, about 36 million people will be working remotely by the end of 2025. This amounts to nearly 22% of the entire workforce—a massive 87% jump from pre-pandemic figures.

While this shift has irrefutable benefits, it also exposes new vulnerabilities. A more distributed network opens up multiple endpoints and increases security concerns across operations. 2020 also witnessed the 5g rollout which made devices more connected than ever before but has also exacerbated the security risks associated with IoT devices.

All things considered, cybersecurity has perhaps never been more crucial. In 2021 the frequency of new cyber-attack incidents is estimated to be one every 11 seconds, almost twice the rate of 2019 (one every 19 seconds). Moreover, the total annual cost borne by the victims of cybercrimes around the world is pegged to be north of US$ 6 trillion by 2021—A figure substantial enough to rival large national GDPs!

Under these circumstances, enterprises must leave no stone unturned when it comes to leveraging the right technologies to secure their digital futures. Traditionally, enterprises have deployed integrated cybersecurity solutions based on legacy, centralized architectures.

But there is a better, more robust way to tackle threats. Edge AI is a system that harnesses AI and ML algorithms and processes the data generated by a local Edge Computing environment locally. It holds the potential to greatly enhance security levels, especially in terms of data privacy due to the lack of a centralized repository. But how can organizations go about implementing and translating Edge AI, a relatively new concept, to tangible business success?

Edge AI: An Introduction

Edge AI transfers the ability to process information to a distributed model rather than the legacy central model. This increases the speed of both data processing and data churning. Edge AI’s distributed model can address privacy requirements and maintain a much stronger operational security posture.

For a remote workforce, Edge AI is a highly efficient and effective cybersecurity solution as it can effectively counter the challenges pertaining to data privacy and security that arise out of having multiple endpoints. In fact, smart enterprises have already woken up to their advantages and the market value of Edge AI is expected to grow from just US$355 million in 2018 to US$ 1.12 trillion in 2023.

Taking the Edge off Security Vulnerabilities

Edge AI combines the productivity and efficiency of automation with the security of edge computing. As the operations are handled in smaller chunks at the individual endpoints, users can incorporate more security capabilities without disrupting overall performance. By adequately addressing data privacy and security concerns, organizations are relatively more confident of their ability to comply with the different regulatory as-well-as standards and including the cybersecurity capabilities, wherein the business teams can take a more aggressive and confident approach towards business growth in a highly secure and compliant environment. The AI capabilities can help automate tasks and accelerate DevOps, which boosts overall productivity.

Of course, there are certain caveats of leveraging Edge AI as well. For an enterprise, Edge computing expands the area of operation by supporting a more distributed form of operations. This means that threat actors can target each edge individually. Although, at the same time, Edge AI-powered distributed frameworks support the standardizing of security postures. Advanced security measures and upgrades thus can be replicated in a uniform manner creating a more secure environment than a centralized model.

Enterprises that seek to implement Edge AI must be careful to account for the capabilities and considerations of both AI and Edge computing. They will do well to adopt and follow a set of guidelines and best practices that ensure a smooth optimal Edge AI implementation.

What to Watch for When Implementing Edge AI

Here is a suggestive list of factors that enterprises must account for while implementing Edge AI.

  • Users must apply basic security best practices with devices involved in edge computing. This includes ensuring the use of well-defined and demarcated levels of protection, access types, robust patch management programs and version update status of programs/OS.
  • Platforms and solutions that are leveraged to enable edge AI must be highly secure as well. Their base capabilities must tie back to a much stronger security posture.
  • Access and roles from administration to processing to consuming data output must be defined clearly. Specific accessibility criteria and privileges for the entire chain including bots and system functions must be defined and secured. Basically any data endpoint within the security chain has to be protected.
  • IT must have clear visibility into the entire chain and respective processes. They need to closely monitor access and privilege elevations so that they can observe and analyze all the information and events, and successfully preempt any kind of malicious activity. Monitoring can add the capability to localize and remediate any suspicious activities. It is especially important because malicious activities traversing from one edge to others can be catastrophic to the entire system.
  • A culture of process monitoring on the edges can help prepare for enterprise disaster recovery (DR) as well. As IT does not have absolute control over the edge, enterprise DR must account for extensive DR planning with special focus on communication, recalibration of DR testing, and network redundancy.

Conclusion

Edge AI can have a direct impact on businesses and their bottom-line. However, organizations must tread carefully around the various complexities and considerations related to implementing Edge AI. Partnering with experienced service delivery teams can help organizations enjoy a smooth and optimal Edge AI implementation experience. Getting Edge AI right can help enterprises navigate the murky and untested waters of virtual work but also help them usher in an era of more secure and productive operations.

Tweets:

  • Edge AI holds the potential to enhance security levels, especially in terms of data privacy as the data is not bound to a centralized repository
  • Edge AI can be a highly efficient solution of remote working as it can counter the myriad challenges pertaining to data privacy and security arising out of multiple endpoints
  • Edge AI blends in the productivity and efficiency capabilities of Automation with the security of edge computing

Authored by:

Renju Varghese,

Fellow and Chief Architect of the Cybersecurity & GRC business, HCL Technologies

The post Edge AI: Changing the Cyber Security Landscape through Scalable and Flexible Data Safety appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Categories
Nasscom Community

Empowering Organizations with Talent-as-a-Service Model

Would you like to keep investing in an employee whom you can never utilize for the organization’s benefit? Or would you like to invest in one able and efficient professional whom you can call in for help whenever you feel the demand?

Under normal circumstances, even if we could afford to have resources on the bench, in a disruptive world like today’s, such a thing is unthinkable. Here arrives the savior of the situation, the TaaS, or the Talent-as-a-Service model. As the name very well demonstrates, TaaS is all about organizations utilizing highly skilled, talented professionals on project-wise demand via a cloud-based platform. Other than anything, adopting a TaaS model leads an organization towards cost-cutting and efficiency enrichment.

For so long, we have seen the startups working with the TaaS model more than any other organization, but today even the MNCs are highly into it. Whatever innovative plans and ideas you think of executing, this model delivers the fastest. This way, the company fund also finds its best value and utilization.

The model in question is known to engage the right talents in fitting projects and make way for learning, but at the same time, it does not snatch away from the professionals their control over their endeavors and time. TaaS builds a collaborative workforce, where both the organization as well as the talent can work compliantly.

Over the last six months of the pandemic, it is not the recruitment process, but the priorities of the organizations and the related strategies have changed. While many have halted their recruitments, some others are only interested in grasping the brightest talents. The situation is unsteady and utterly risky today, and TaaS utilization proves to be the best strategy for a sound business.

As far as the talented professionals are concerned, the ones working as a part of the TaaS model find it working out just fine for them because –

  • they can do more meaningful work in a lesser stressful ambiance
  • get great exposure and learning opportunities from all the organizations they work with
  • have a flexible work schedule
  • can work from any location (which is the real need of the hour)

The pandemic has fundamentally transformed the entire work milieu. This transformation is and will be a one-of-a-kind experience. Amid every uncertainty floating presently, the only aim in the economic realm is to keep the businesses pertinent and competitive. For talented professionals, it has never been a better phase to work satisfactorily on their terms, utilize their skillsets, make money, and gain pools of experience.

The post Empowering Organizations with Talent-as-a-Service Model appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Categories
Nasscom Community

Data analytics is key for driving Airline industry transformation

COVID-19 has brought about numerous challenges for industries across the board. But this pandemic has also forced us to think out-of-the-box and come up with innovative ways to overcome the challenges that come along with a global health crisis like this one.

It could be because I work in the data and analytics space, but I see an opportunity here. I believe that data can help us combat the spread of deadly viruses and diseases. If we can cross-leverage data from industries such as travel and healthcare, we could potentially preempt the spread of viruses, and their point of origin, and provide advance warning to passengers that will allow them to make informed travel decisions. This would also enable airlines to plan their travel routes accordingly, to avoid unforeseen last-minute cancellations and schedule changes.

Big data can help make sense of large, integrated data-sets, and provide a Single Source of Truth (SSOT). This will equip decision-makers with quicker, and more pertinent insights, that can have a more overarching impact on the business. Staying within the bounds of data privacy laws, a collaboration between the healthcare sector and the travel industry could deliver more value to the customer.

If airline companies can access electronic health records (EHR) of passengers, even at an aggregate level, it would allow them to tailor the flight experience for their passengers accordingly. Healthcare institutions on the other hand can have access to patient travel history aggregated from airline carriers, which will help them offer better diagnosis and treatment. Confidential data from both sides can be exchanged using a unique or composite key, and it can be encrypted using cryptographic technologies such as blockchain, to prevent the loss or misuse of data.

There are tremendous possibilities that can be explored, if we can get the travel and healthcare systems to talk to each other. But let’s delve a little deeper into how the three main pillars of analytics (descriptive, predictive, and prescriptive) can specifically help drive transformations in the airline industry:

Descriptive Analytics (historical comparison):

Airlines can offer various bespoke services to their passengers, if they had access to the passengers’ health records. For instance, while booking the ticket:

    • Airlines can make in-flight meal and seat recommendations, based on the passenger’s dietary restrictions (food allergies), chronic disorders, and other pre-existing conditions
    • They can offer special assistance to passengers who are suffering from serious ailments, on the flight
    • Passengers can be forewarned about epidemics, air quality index, and other health hazards at the destination
    • Airline carriers would also be better equipped to provide emergency care to passengers’ mid-air or at the airport, if they can access information such as blood group, chronic conditions, etc.

Predictive and Prescriptive Analytics (predictions and charting future course):

If airlines could mine data and use modelling techniques to identify patterns and predict future outcomes, it would help them to significantly optimize their resources and boost revenues. It would also allow them to ensure that they work with local governments to nip possible virus outbreaks, in the bud. Here are a few possibilities:

    • Airlines can provide travel histories of passengers to the local health departments (in destination countries), to help identify people travelling from regions that are combating a disease, or virus
    • Flying restrictions can be imposed on certain individuals until they get themselves tested, if their travel history suggests that they could be asymptomatic carriers of a pathogen
    • Airline companies can assess the risks associated with flying through certain routes based on several regional parameters, and decide if they want to alter their flight schedules
    • Push notifications can be sent to passengers who have installed the airline company’s mobile app, warning them about possible health risks in their destination country, based on their existing health conditions (for example – warning a passenger suffering from a breathing disorder, about the quality of the air in their arrival city)

These were some use cases that demonstrate how data can potentially save passenger lives, curb the spread of deadly pathogens, and allow airline companies to optimize their operations and revenue. But if we scratch beneath the surface, data can yield numerous opportunities to help transform the airlines industry and improve the passenger experience. The knowledge that the airline carrier you are flying with is fully equipped to provide assistance in case you face a medical emergency, can be very reassuring. I know at least I will feel much safer when I fly next time.
This blog was contributed by Manoj Panicker from Sabre GCC, Bengaluru.

The post Data analytics is key for driving Airline industry transformation appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Categories
Nasscom Community

6 Things to Remember When Preparing Big Data for Advanced Analytics

Studies indicate that organizations are finding it difficult to realize the full potential of data. Only about half of the organizations think they are able to use data and analytics for competitive purposes.

By Prashant PawarHead of Data and Cloud CoE, Nihilent
Prabith Kalathil Regional Head, Africa, Nihilent

“Information is the oil of the 21st century, and analytics is the combustion engine” -Peter Sondergaard, Former Senior Vice President, Gartner

The quote above highlights the importance of big data management and analytics. Many organizations have ambitious plans for analytics and ML. Overall the investments in big data and analytics are increasing every year. However, studies indicate that organizations are finding it difficult to realize the full potential of data. Only about half of the organizations think they can use data and analytics for competitive purposes. And even less than that think they are a data-driven organization and are getting results from the data and AI investments. 55% of data collected by organizations is never used. Why is this happening?

On the other hand, we came across organizations that have been very successful in using data for their business needs. When we look at those examples one thing that stood out was that technology was not the main barrier here. There are several open-source and proprietary platforms available today. Hadoop led the technology landscape in the last decade and then many other options emerged in specific areas like MPP databases, NoSQL databases, data lakes, advanced analytics, stream analytics, BI, ML, and so on. We observed is that the successful big data implementations have high levels of seamless integration between data assets, are agile enough to respond to the changing business needs, and enable self-service BI and analytics for the end-user. These attributes of seamless integration, agility, self-service BI are achieved by focusing on six key data disciplines.

#1: High-Speed Data Acquisition and Processing

There are many options for how you can do data acquisition, extraction, and ingestion into the big data platform. The choice will largely depend on factors like the frequency with which you want to capture the incoming data, whether the data is coming in batches or real-time, whether to employ a push or pull model, etc. Irrespective of the tool or approach you choose, how well your data platform performs in terms of speed, will ultimately decide if users will use the platform widely.

#2: Metadata Management and Data Catalog

Metadata helps us decode three different perspectives. One is, it will help us to create very accurate and consistent reports. Metadata is information about your data. If we know the metadata very well, then, we can check where the data is coming from in our reports. The second use of metadata is, it allows the end-user to find data. This is important, especially in a big data platform. Because in a big data platform we end up collecting lots of data, thousands of data sets are collected from the data source daily. It becomes very difficult for a data scientist or a data analyst who wants to find the data in a large data lake or a data warehouse that you have created. The third use is, if you combine it with other elements it will also help you to track the data lineage as well.

# 3: Ensuring Data Quality

We are all aware that if we do not maintain data quality, then the data platform soon reflects a ‘garbage in garbage out’ type of scenario. Therefore, maintaining data quality is paramount. There are many different thoughts about who is responsible for data quality. We tend to agree with the suggestion that data quality is the responsibility of the data owner. However, the data platform should have some tools available to check the quality of data when it is integrated on the central platform.

# 4: Master Data Management (MDM)

The next important point is Master Data Management (MDM). It’s about maintaining the master list, whether it is the products, vendors, suppliers, or customers. Having a master list across the organization helps to create accurate analytics. Usually, your organization will require standardized master lists in order to bring consistency in reporting and analytics. These master lists act as a single source of truth across the organization. The master lists will be consistent across the data platform. You can consolidate master lists by (1) matching and merging, (2) data standardization, (3) data consolidation.

# 5: Data Security

The fifth ability is about data security and access control. When we bring data together in a central storage area, the data owners demand the highest level of data security especially when that data is personal or financial data. Data owners will not give you the data if there is no guarantee that data is going to be secured. It has to be protected and secured from unauthorized access by using measures like user authentication, User Access Control using RBAC or Item level security, data encryption, and network security. These security measures should be implemented in all components of the big data platform.

# 6: Data Lineage

The last important aspect is the data lineage. It is about tracing data back to its origin. In order to trust the data used for BI and analytics, users will demand to know the flow of their data, where it originated, who had access to it, which changes it underwent, and when. They would want to know where it resided throughout the organization’s multiple data systems. Hence data lineage is an important aspect.

These are the 6 key areas important to make your data platform scalable, agile, searchable, secure, and traceable.

 

The post 6 Things to Remember When Preparing Big Data for Advanced Analytics appeared first on NASSCOM Community |The Official Community of Indian IT Industry.

Page 2 of 53
1 2 3 4 53