Thursday, 27 December 2018

Informatica MDM – Suspect Duplicate Process (SDP) Approach


A master data management (MDM) system is installed so that the core data of an organization is secure,  is accessible by multiple systems as and when required and does not have multiple copies floating in the system, in order to have a single source of truth. A solid Suspect Duplicate Process is required in order to achieve the 360 degree view of an entity.

The concept of Suspect Duplicate Processing represents the broad category of activities related to identifying entities that are likely duplicates of each other. Suspect duplicate processing is the process of searching for, matching, creating associations between and, when appropriate, merging data for existing duplicate party records in the system.

To achieve this functionality, Informatica MDM has come up with its own Suspect Duplicate Processing (SDP) approach. An organization based on its use case can opt any of the following two approaches:


  • Deterministic Matching Approach
  • Fuzzy Matching Approach


Deterministic Matching Approach

Deterministic Matching uses a series of rules, like nested if statements, to run a series of logical tests on the data sets. This is how we determine relationships, hierarchies, and households within a dataset. Deterministic matching seeks a clear “Yes” or “No” result on each and every attribute, based on which we define whether:


  • Two records are duplicates
  • should be resolved by a data steward or
  • Two unique entities.


It doesn’t leave any room for error and provides the result in an ideal scenario. But most of the data in organizations is far from an ideal scenario. These are the cases when the Fuzzy Matching Approach of Informatica comes handy.

Learn more at http://www.infotrellis.com/informatica-mdm-fuzzy-matching/

Tuesday, 25 December 2018

Mastech InfoTrellis - Experts in Big Data Analytics

Mastech InfoTrellis’ diverse expertise in the Big Data space, has helped to assist global enterprises in their Big Data initiatives

Big Data Analytics Hub

Mastech InfoTrellis offers managed Big Data Analytics Hub Solution Centered on Hadoop, which enables customers to consolidate multi-channel data of various formats into a single source. Big Data Analytics Hub enables self service analytics by different business functions.

AllSight – Customer Intelligence Management

AllSight Customer Intelligence Management System which delivers Enterprise Customer 360 by ingesting structured and unstructured data from disparate data sources across the organization.

IBM Big Data Solutions

IBM Big Data Solutions combine open source Hadoop and Spark for the open enterprise to cost effectively analyze and manage big data. With BigInsights, you spend less time creating an enterprise-ready Hadoop infrastructure, and more time gaining valuable insights. IBM provides a complete solution, including Spark, SQL, Text Analytics and more to scale analytics quickly and easily.

Learn more at http://www.infotrellis.com/big-data/

Saturday, 22 December 2018

Data Management and IBM IIS Tools

As per a study conducted by a leading market research and advisory company the data that we have generated in the past two years is many times more than that we generated in over two decades. It has not just multiplied, but have also become complex, varied and is being generated at much more rate than it ever was. These factors present a data integration challenge to the industries and businesses to be able to better utilize their data for help building strategies, provide services, introduce policy regulations such that their business is empowered to bridge or completely meet the gap for that matter between data and analytics.
IBM has always been innovative, technology-driven and in fact, they pioneer in data integration and management technologies. They have always provided the business with the right set of tools and IBM MDM (Master Data Management) is the best example of that. Besides MDM, IBM also has IIS (InfoSphere Information Server) in its quiver to target data integration and management challenges that almost every line of business in this age encounters.
This blog aims to provide an outlook on the IBM IIS suite and how it can empower your business data integration demands for better resource utilization and finding the right set of tools to address key business challenges.

http://www.infotrellis.com/data-management-ibm-iis-tools/

Wednesday, 19 December 2018

Best Practices in Data Validation

Data Quality is the buzz word in the digital age.

What is data quality and why is it so important?

“Data quality” is the term that is probably hidden but plays an important role in many streams. Data plays a vital role in acquiring a market place, especially in enterprise data management stream.

Data Quality Examples

Following are some examples which emphasize the need for data quality.
  • A customer shouldn’t be allowed to enter his age where he has to mention his marital status.
  • When a customer enters a store, there is a high possibility that he might miss out his original details to be filled up with the forms, some of it can be in a hurry not mentioning a correct phone number.
  • There is also a possibility of the billing staff to wrongly enter the store address as default in place of the customer address which contributes to a bad quality data that gets persisted in the system.
This data may be crucial as the customer might not just be a Guest customer and the customers’ viable interest towards the store becomes obscure.
This blog post speaks on Data Quality, the significance of Data Quality, business impacts, best practices to be followed, and Mastech InfoTrellis’ specialization in Data validation.
http://www.infotrellis.com/best-practices-data-validation/

Monday, 10 December 2018

Why Big Data in Healthcare is so required

“Data analytics” refers to the practice of taking masses of aggregated data and analyzing them to draw important insights and information contained in it. This process is increasingly aided by new software and technology that helps examine large volumes of data for hidden information that can help us in many areas and healthcare is one of those areas.

80% of all healthcare information is unstructured data which is so vast and complex that it needs specialized methods and tools to make meaningful use of the data. The new and emerging technologies like artificial intelligence (AI), machine learning, and predictive analytics are bringing in powerful tools for healthcare technologists and thought leaders to capture these data and process it effectively and efficiently for the complete transformation of the healthcareindustry.Physician decisions are winding up increasingly prove based, implying that they depend on expansive swathes of research and clinical information rather than exclusively their tutoring and expert sentiment. This new treatment state of mind implies there is a more prominent interest for big data analytics in medicinal services offices than at any other time. There is almost certainly that big data has developed as a defining moment changer for the healthcare industry to enable it to advance to another level.

Read full article at http://www.infotrellis.com/big-data-analytics-augmented-patient-care/

Why Big Data in Healthcare is so required

“Data analytics” refers to the practice of taking masses of aggregated data and analyzing them to draw important insights and information contained in it. This process is increasingly aided by new software and technology that helps examine large volumes of data for hidden information that can help us in many areas and healthcare is one of those areas.

80% of all healthcare information is unstructured data which is so vast and complex that it needs specialized methods and tools to make meaningful use of the data. The new and emerging technologies like artificial intelligence (AI), machine learning, and predictive analytics are bringing in powerful tools for healthcare technologists and thought leaders to capture these data and process it effectively and efficiently for the complete transformation of the healthcareindustry.Physician decisions are winding up increasingly prove based, implying that they depend on expansive swathes of research and clinical information rather than exclusively their tutoring and expert sentiment. This new treatment state of mind implies there is a more prominent interest for big data analytics in medicinal services offices than at any other time. There is almost certainly that big data has developed as a defining moment changer for the healthcare industry to enable it to advance to another level.

Read full article at http://www.infotrellis.com/big-data-analytics-augmented-patient-care/

Overview of Informatica PowerCenter Web Service

Web Services Overview:
Web Services are services available over the web that enables communication and provide a standard protocol for communication. To enable the communication, we need a medium (HTTP) and a format (XML/JSON).

There are two parties to the web services, namely Service Provider and Service Consumer. A web service provider develops/implements the application (web service) and makes it available over the internet (web).  Service Provider publishes an interface for the web services that describes all the attributes of the web service. Service Consumer consumes the web service. For the Consumer to consume the web service, the consumer has to know the services available, request and response parameters, how to call the services and so on.

Hence we can define Web Service as a standardized way of integrating web desk applications using XML, SOAP, WSDL and UDDI open standards over an internet protocol backbone. XML is used to tag the data. SOAP is used to transfer the data. WSDL is used for describing the services available and UDDI is used for listing what services are available.

Learn more at, http://www.infotrellis.com/how-to-access-informatica-powercenter-as-a-web-service/

Tuesday, 4 December 2018

Informatica Power Center Solutions

Promote automation, reuse and agility with the industry's only full integrated end-to-end enterprise data integration platform.

Informatica's modern data integration infrastructure combines advanced hybrid data integration capabilities and centralized governance with flexible self-service business access for analytics. By providing a robust integrated codeless environment, teams can collaboratively connect systems and transform and integrate data at any scale and any speed.

Read full article at http://www.infotrellis.com/informatica-data-integration/

Monday, 3 December 2018

Big Data Analytics & Data Management Services

Mastech InfoTrellis’ diverse expertise in the Big Data space, has helped to assist global enterprises in their Big Data initiatives

Big Data Analytics Hub
Mastech InfoTrellis offers managed Big Data Analytics Hub Solution Centered on Hadoop, which enables customers to consolidate multi-channel data of various formats into a single source. Big Data Analytics Hub enables self service analytics by different business functions.

AllSight — Customer Intelligence Management
AllSight Customer Intelligence Management System which delivers Enterprise Customer 360 by ingesting structured and unstructured data from disparate data sources across the organization.

IBM Big Data Solutions
IBM Big Data Solutions combine open source Hadoop and Spark for the open enterprise to cost effectively analyze and manage big data. With BigInsights, you spend less time creating an enterprise-ready Hadoop infrastructure, and more time gaining valuable insights. IBM provides a complete solution, including Spark, SQL, Text Analytics and more to scale analytics quickly and easily.

Read full story at http://www.infotrellis.com/big-data/

Sunday, 2 December 2018

Enterprise Data Integration Services

Enterprise Data Integration Services

Using niche technologies, Mastech InfoTrellis enables customers to extract, transform and load data from disparate source systems to centralized data repositories like Master Data Management Hub, Big Data and Analytics Hub.


  • Etl performance tuning
  • Metadata management
  • Data quality monitoring
  • Cross platform integration
  • Data modelling
  • Data profiling


Our Solutions

  • Informatica Intelligent Data Integration
  • Informatica Intelligent Cloud Services
  • Collibra Data Governance


Learn more at http://www.infotrellis.com/enterprise-data-integration/

Friday, 30 November 2018

Automate Informatica Data Quality (IDQ)

Data Quality – Overview
Data Quality is the process of understanding the quality of data attributes such as data types, data pattern, existing values, and so on. Data quality is also about capturing the score of an attribute based on some specific constraints. For example, get the count of records for which the attribute value is NULL, or find the count of records for which a date attribute does not fit into the specified Date Pattern.

Managing your Data Quality
This means that we can weigh the quality of data to any extent irrespective of the available data being good or bad. This Data Quality report can be captured with the complete data details, at record level or even at the attribute level. Using this report, business can identify the quality of data and make out how it can be used to help / benefit the customer. A plan can also be worked out to enhance the quality of data by applying business rules and correcting the required information based on the business needs.

This blog post aims at bringing out the significance of data quality, data quality report generation, steps involved in automation of the data quality report using the scheduler feature of Informatica IDQ.

Deriving Quality Data
We have tools in the market to generate these Data Quality reports based on the input data we provide with configuration of some business specifications. An important solution provider in the market for Data Quality report generation is Informatica IDQ which is formulated to generate profiling reports and Data Quality reports.

Read full article at http://www.infotrellis.com/automate-data-quality-informatica-idq/

Thursday, 22 November 2018

Informatica MDM Solution - Mastech Infotrellis

A master data management (MDM) system is installed so that the core data of an organization is secure,  is accessible by multiple systems as and when required and does not have multiple copies floating in the system, in order to have a single source of truth. A solid Suspect Duplicate Process is required in order to achieve the 360 degree view of an entity.

The concept of Suspect Duplicate Processing represents the broad category of activities related to identifying entities that are likely duplicates of each other. Suspect duplicate processing is the process of searching for, matching, creating associations between and, when appropriate, merging data for existing duplicate party records in the system.

To achieve this functionality, Informatica MDM has come up with its own Suspect Duplicate Processing (SDP) approach. An organization based on its use case can opt any of the following two approaches:


  • Deterministic Matching Approach
  • Fuzzy Matching Approach


Deterministic Matching Approach

Deterministic Matching uses a series of rules, like nested if statements, to run a series of logical tests on the data sets. This is how we determine relationships, hierarchies, and households within a dataset. Deterministic matching seeks a clear “Yes” or “No” result on each and every attribute, based on which we define whether:


  • Two records are duplicates
  • should be resolved by a data steward or
  • Two unique entities.


It doesn’t leave any room for error and provides the result in an ideal scenario. But most of the data in organizations is far from an ideal scenario. These are the cases when the Fuzzy Matching Approach of Informatica comes handy.

Read full article at http://www.infotrellis.com/informatica-mdm-fuzzy-matching/

Tuesday, 20 November 2018

Why is Master Data Management important?

Mastech InfoTrellis offers best of breed Master Data Management Services enabling Customers to harness the power of their Master Data. Mastech InfoTrellis has successfully delivered Master Data Management Projects time and again over the past decade.

Performance tuning
Production support
Health check
Solution architecture
Needs assessment
Program strategy & roadmap
Solution upgrade
Design and development

Our Solutions

IBM InfoSphere Master Data Management
Cloud Customer 360 For Sales Force
Informatica Intelligent Master Data Management
IBM PIM For Manufacturing

Learn more at http://www.infotrellis.com/master-data-management/

Saturday, 17 November 2018

Best Practices for Master Data Management



Mastech InfoTrellis offers best of breed Master Data Management Services enabling Customers to harness the power of their Master Data. Mastech InfoTrellis has successfully delivered Master Data Management Projects time and again over the past decade.

For more information https://bit.ly/2TmCvCj

Tuesday, 23 October 2018

Interfacing Virtual MDM through DataStage

MDM Connector stage is a key to open the door of IBM Virtual MDM. Yes, we can manipulate the data in MDM (MDM refers to IBM Virtual MDM in this post) using the MDM Connector stage which was introduced in IBM DataStage v11.3.
We know that loading data into MDM is not an easy task since it involves many tables and the relationship among the tables should be maintained properly, otherwise will end up dealing with junk not with the data. MDM Connector stage makes this task simpler by allowing us to configure everything in the single configuration window.
This blog post details on how the basic operations (read/write) on data can be performed using the Connector stage in v11.5.

http://www.infotrellis.com/interfacing-virtual-mdm-datastage/

Monday, 15 October 2018

Best Master Data Management Strategy

Mastech InfoTrellis offers best of breed Master Data Management Services enabling Customers to harness the power of their Master Data. Mastech InfoTrellis has successfully delivered Master Data Management Projects time and again over the past decade.


IBM InfoSphere Master Data Management

IBM InfoSphere Master Data Management (MDM) manages all aspects of your critical enterprise data, no matter what system or model, and delivers it to your application users in a single, trusted view. Provides actionable insight, instant business value alignment and compliance with data governance, rules and policies across the enterprise.

Cloud Customer 360 For Sales Force

Informatica Cloud Customer 360 for Salesforce eradicates duplicate, inaccurate, and incomplete account and contact records. It provides clean, trusted data, increases Salesforce user adoption, and boosts ROI.

Informatica Intelligent Master Data Management

A complete master data management solution addresses the critical business objectives digital organizations face. Informatica MDM offers the only true end-to-end solution, with a modular approach to ensure better customer experience, decision making and compliance.

IBM PIM For Manufacturing

Establish a single view of product information for strategic business initiatives with IBM Product Information Management solution. IBM PIM solution enables Service Oriented Architecture, provides a flexible data model, aligns to existing business processes and is scalable to suit to the growing product landscape of an organization.

Contact:
9390 Research Blvd., Suite 330
Austin, TX 78759
United States
Phone: +1-512-358-1396
Website: http://www.infotrellis.com

Tuesday, 11 September 2018

Big Data Management System

Mastech InfoTrellis’ diverse expertise in the Big Data space, has helped to assist global enterprises in their Big Data initiatives

Big Data Management System

Our Solutions

Big Data Analytics Hub
Mastech InfoTrellis offers managed Big Data Analytics Hub Solution Centered on Hadoop, which enables customers to consolidate multi-channel data of various formats into a single source. Big Data Analytics Hub enables self service analytics by different business functions.

AllSight – Customer Intelligence Management

AllSight Customer Intelligence Management System which delivers Enterprise Customer 360 by ingesting structured and unstructured data from disparate data sources across the organization.

IBM Big Data Solutions

IBM Big Data Solutions combine open source Hadoop and Spark for the open enterprise to cost effectively analyze and manage big data. With BigInsights, you spend less time creating an enterprise-ready Hadoop infrastructure, and more time gaining valuable insights. IBM provides a complete solution, including Spark, SQL, Text Analytics and more to scale analytics quickly and easily.


Monday, 3 September 2018

IBM MDM BatchProcessor – Tips for better throughput

MDM BatchProcessor is a multi-threaded J2SE client application used in most of the MDM implementations to load large volumes of enterprise data into MDM during initial and delta loads. Oftentimes, processing large volumes of data might cause performance issues during the Batch Processing stage thus bringing down the TPS (Transactions per Second).

Poor performance of the batch processor often disrupts the data load process and impacts the go-live plans. Unfortunately, there is no panacea available for this common problem. Let us help you by highlighting some of the potential root causes that influence the BatchProcessor performance. We will be suggesting remedies for each of these bottlenecks in the later part of this blog.

Infrastructure Concerns
Any complex, business-critical Enterprise application needs careful planning, well ahead of time, to achieve optimal performance and MDM is no exception. During development phase it is perfectly fine to host MDM, DB Server and BatchProcessor all in one physical server. But the world doesn’t stop at development. The sheer volume of data MDM will handle in production needs execution of a carefully thought-out infrastructure plan. Besides, when these applications are running in shared environments Profiling, Benchmarking and Debugging become a tedious affair.

CPU Consumption
BatchProcessor can consume lot of precious CPU cycles in most trivial of operations when it is not configured properly. Keeping an eye for persistently high CPU consumption and sporadic surges is vital to ensure CPU is optimally used by BatchProcessor.

Deadlock
Deadlock is one of the frequent issues encountered during the Batch Processing in multi-threaded mode. Increasing the submitter threads count beyond the recommended value might lead into deadlock issue.

Stale Threads
As discussed earlier, a poorly configured BatchProcessor might open up Pandora’s Box. Stale threads can be a side-effect of thread count configuration in BatchProcessor. Increasing the submitter threads, reader and writer threads beyond the recommended numbers may cause some of the threads to wait indefinitely thus wasting precious system resources.

100% CPU Utilization
“Cancel Thread” is one of the BatchProcessor daemon threads, designed to gracefully shutdown BatchProcessor when the user intends to. Being a daemon thread, this thread is alive during the natural lifecycle of the BatchProcessor. But the catch here is it hogs up to nearly 90% of CPU cycles for a trivial operation thus bringing down the performance.

Let us have a quick look at the UserCancel thread in BatchProcessor client. The thread waits for user interruption indefinitely and checks for the same every 2 seconds once while holding on the CPU all the time.

Read full article at https://bit.ly/2Nvx3Nh

Monday, 27 August 2018

MDM for Regulatory Compliance in the Banking Industry

Banking Regulations – Overview
Managing regulatory issues and risk has never been so complex. Regulatory expectations continue to rise with increased emphasis on the institution’s ability to respond to the next potential crisis. Financial Institutions continue to face challenges implementing a comprehensive enterprise-wide governance program that meets all current and future regulatory expectations. There has been a phenomenal rise in expectations related to data quality, risk analytics and regulatory reporting.
Following are some of the US regulations that MDM and customer 360 reports can be used for compliance:
FATCA (Foreign Account Tax Compliance Act)
FATCA was enacted to target non-compliance by U.S. taxpayers using foreign accounts. The objective of FATCA is the reporting of foreign financial assets. The ability to align all key stakeholders, including operations, technology, risk, legal, and tax, is critical to successfully comply with FATCA.
OFAC (Office of Foreign Asset Control)
The Office of Foreign Assets Control (OFAC) administers a series of laws that impose economic sanctions against hostile targets to further U.S. foreign policy and national security objectives. The bank regulatory agencies should cooperate in ensuring financial institutions comply with the Regulations.
FACTA (Fair and Accurate Credit Transactions Act)
Its primary purpose is to reduce the risk of identity theft by regulating how consumer account information (such as Social Security numbers) is handled.
HMDA (Home Mortgage Disclosure Act)
This Act requires financial institutions to provide mortgage data to the public. HMDA data is used to identify probable housing discrimination in various ways.
Dodd Frank Regulations
The primary goal of the Dodd-Frank Wall Street Reform and Consumer Protection Act was to increase financial stability. This law places major regulations in the financial industry.
Basel III
A wide sweeping international set of regulations that many US banks must adhere to is Basel III. Basel III is a comprehensive set of reform measures, developed by the Basel Committee on Banking Supervision, to strengthen the regulation, supervision and risk management of the banking sector.
What do banks need to meet regulatory requirements?
To meet the regulatory requirements described in the previous section, Banks need an integrated systems environment that addresses requirements such as Enterprise-wide data access, single source of truth for customer details, customer identification programs, data auditability & traceability, customer data synchronization across multiple heterogeneous operational systems, ongoing data governance, risk and compliance reports.

What You May Be Missing by Not Monitoring Your MDM Hub

Organizations spend millions of dollars to implement their MDM solution. They may have different approaches (batch vs. real time; integrated customer view vs. integrated supplier view etc.) – but in general they all expect to get a “one version of the truth” view by integrating different data sources and then providing that integrated view to a variety of different users.

After the completion and successful testing of the MDM implementation project, companies sit back and enjoy the benefits of their MDM hub – and more often than not don’t even think about looking under the hood. It never occurs to them that they could be trying to gain insights into what’s happening inside that MDM hub by asking questions like

–          How is the data quality changing?

–          What are the primary activities (in processing time) inside the MDM hub?

–          How are service levels changing?

However, organizations change, people change, requirements change – impacting what is happening inside the MDM Hub. Such changes can open up significant opportunities for an organization – but without doing any sort of investigation that opportunity is typically not recognized.

Here are two examples – diagnosed through the use of an MDM audit tool:

–          The company’s MDM Hub had approximately 100,000 incorrect customer addresses. These addresses were used for regular mailings; the mailings generated (in case of correct address) incremental revenues. Impact on the business related to just one mailing:

$400K wasted on the mailing cost ($4 is the conservative mailing cost per person – for postage, printing of the mailer etc.)
$100K of immediately lost revenues (as past data shows that one in 50 customers spends about $50 immediately following the mailing)
The longer term revenue lost was not assessed, but was estimated to be well over $400K
The opportunity: Cost saving of $400k and revenue increase of $500K or more
–          At a different company, by analyzing data processed by week the resulting report was able to determine that the number of new customers processed was declining by 1-2% every week – starting about 6 weeks before the audit was conducted. A deeper review of the audit report suggested that

The original service levels related to customer file changes had been getting worse and worse over that same time period

As customer file changes (as per the audit report) took over 85% of the total processing time, the slower processing lead to less time available for new customer processing

This initial diagnostic was confirmed by the client – they had a slowly growing backlog of new customer files

Ultimately the audit was able to highlight which input data source had been causing the slowdown, allowing the company to resolve the problem at its source

Business impact: a major risk (very significant slowdown in new customer set up) was eliminated before it became a real problem

Read full story at https://bit.ly/2PTp0Ix