Inputs:

OpenAPI Specification

Swagger OpenAPI 3.0

MPI $patient match example

E-Health Apiary (GET/POST method)

Confluence example for Patient Merge and Link

• Step2 - merger operation

I have completed the following:

• Terminal (local frameworks)
• node js
• express js
• Codecademy course – @Swagger
• Yaml, Markdown, JSON
• OpenAPI 3.0 guide

Right now, I am in the middle of the REST API spec for $Link MPI Kodjin

Here is a link — Swagger Editor

There are a few questions to talk about:

1. Let's begin with Critical thinking
2. Planning and designing the API $Link

{
	"Patient": {
		"id": {
			"_value": "01"
		},
		"identifier": {
			"use": {
				"_value": "official"
			},
			"type": {
				"coding": {
					"system": {
						"_value": "http://www.hospital-a/localid"
					},
					"code": {
						"_value": "HospitalA"
					},
					"display": {
						"_value": "Hospital A"
					}
				},
				"text": {
					"_value": "Hospital A"
				}
			},
			"system": {
				"_value": "urn:oid:2.16.840.1.113883.3.72.5.9.1"
			},
			"value": {
				"_value": "1000000001"
			},
			"assigner": {
				"display": {
					"_value": "Hospital A"
				}
			}
		},
		"name": {
			"family": {
				"_value": "WASHINGTON"
			},
			"given": {
				"_value": "MARY"
			}
		},
		"gender": {
			"_value": "female"
		},
		"birthDate": {
			"_value": "1954-07-04"
		},
		"_xmlns": "http://hl7.org/fhir"
	}
}

https://jsonformatter.org/xml-to-json

How to add components?

How to apply:

http://hl7.org/fhir/valueset-issue-type.html

https://www.hl7.org/fhir/valueset-link-type.html

Input: Yaml official website

https://support.smartbear.com/swaggerhub/docs/tutorials/openapi-3-tutorial.html

Install Swagger editor:

https://github.com/swagger-api/swagger-editor/blob/master/README.md

https://editor.swagger.io/

OpenAPI - спецификация

Swagger - инструментарий использующий спецификацию OpenAPI

Swagger allows you to describe the structure of your APIs so that machines can read them. The ability of APIs to describe their own structure is the root of all awesomeness in Swagger. Why is it so great? Well, by reading your API's structure, we can automatically build beautiful and interactive API documentation. API Development for Everyone.

• Allow machines/tools to integrate with API
• Allow humans to implement API code
• Allow humans to read and generate API documentation and test cases

The OpenAPI Initiative (support specification) provides an open-source, technical community, within which industry participants may easily contribute to building a vendor-neutral, open specification for providing technical metadata for REST APIs.

Swagger - set of tools build around implementation of the OpenAPI specification, tjere are few tools like UI, editor, SwaggerHub, etc.

What's New?
BASIC:
Improved reusability
Parameter changes
Improved examples
Content negotiation support

ADVANCED
Support for describing callbacks
Links to express relationships between
operations
Enhanced security definitions

Parameters
Query (users?id=3)
Path (users/keshav.vasudevan)
Header (X-customheader=23)
Cookies (cookie: debug=0)

RequestBody
Further serialization supported by defining media types

SwaggerHub tutorial

Mixpanel ingests petabytes of event data over the network from the mobile, browser, and server-side clients. Due to unreliable networks, clients may retry events until they receive a 200 OK message from Mixpanel. Although this retry strategy avoids data loss, it can create duplicate events in the system. Analyzing data with duplicates is problematic because it gives an inaccurate picture of what happened and causes Mixpanel to diverge from other client data systems that we may sync with, such as data warehouses. This is why we care deeply about data integrity.

Today, we’re excited to share our solution to deduplicating event data at petabyte scale.

Requirements

To address this problem, we needed an approach that was:

• Scalable: Scales to an ingestion volume of 1M+ events/sec
• Cost-Efficient: Optimizes cost/performance overhead to ingestion, storage, and query
• Retroactive: Identifies duplicate events sent arbitrarily later
• Lossless: Preserves duplicates to rollback in case of misconfiguration
• Maintainable: Minimizes operational overhead

State of the art: Ingestion-Time Deduplication

The industry has a lot of creative approaches to solve the deduplication problem. The central theme involves architecting an infrastructure that performs deduplication at the ingestion layer. Customers send a unique identifier, $insert_id as a property on each event. The deduplication infrastructure stores the $insert_id for all the events within a limited retention window (e.g. 7 days) and checks it against every new event for duplicate identification. A key-value store like sharded RocksDB or Cassandra is often used for storage. The lookup costs in the storage can be improved using a bloom filter. This kind of architecture ensures that the duplicates are weeded out of the system at its entry point.

However, this approach fails to meet our requirements for the following reasons:

• ✅ Scalable: Sharded key-value stores can scale horizontally
• ❌ Cost-Efficient: Requires a separate data store and infrastructure for duplicates
• ❌ Retroactive: Can only capture duplicates in a limited retention window
• ❌ Lossless: Drops data at ingestion, so it’s impossible to rollback
• ❌ Maintainable: Dedupe becomes an additional service that must be up 24×7

Our Approach

We architected a solution that satisfies all of our requirements, by ingesting all events and deduplicating them at read-time. A simple approach to read-time deduplication would be to build a hashtable of all $insert_ids on each query; however, this would add non-trivial overhead to our system. But before describing our solution, let’s review a few key aspects of our architecture.

Mixpanel Architecture

(Project, User, Time)-Based Sharding

Mixpanel’s analytics database, Arb, shards its data files by project, user and event time. This allows us to ensure that all data for a given user is co-located, so that behavioral queries can run across many users simultaneously over relevant portions of time.

Lambda Architecture

In Arb, all the events are written to append-only files, which are periodically indexed (compaction) into columnar files in the background. Append-only files are indexed when they reach either a size or age threshold. This approach ensures that queries are both real-time and efficient, by scanning both small, real-time, append-only files and the large, historical, indexed files.

We leveraged these two aspects of our architecture to make read-time deduplication efficient. By first principles, event duplicates have the following properties:

1. Event duplicates belong to the same project
2. Event duplicates belong to the same user
3. Event duplicates belong to the same event-time

We glean the following key insights from these fundamentals:

• We can scope down the search space for event duplicates to the project, user and day — ie, to a single Arb shard.
• We can minimize the overhead of deduplication by amortizing it alongside our lambda architecture to maintain both real-time and efficient queries.

These insights lead to a solution that satisfies all of our requirements.

Deduplication Architecture

Deduping within Mixpanel infrastructure happens both at indexing time and query time.

Our indexer maintains an in-memory hashset by $insert_id for all the events from the files that are being indexed. If it sees a hit on an event, that event is marked as a duplicate by setting a bit on the event in the indexed format. This process has minimal overhead since indexing occurs at the fine-grained shard level.

At query-time, due to our lambda architecture, we scan both indexed files and append-only files. For indexed files, we can check if the duplicate bit is set, and if so, skip processing the event. For the small, append-only files, queries do hash-based deduping on $insert_id. This allows us to be both real-time and efficient, leveraging the power of the lambda architecture.

Performance

From our experiments, we found that indexing of files with 2% duplicates has a time overhead range of 4% to 10%. This does not have any direct impact on our user experience, as indexing is an offline process.

For query-time, we found that reading an extra bit for every event adds around 10ns to the reading of data. This is close to a 2% increase in the query time because of the additional column. Reading 10 million events adds a time overhead close to 0.1 seconds (100ms). For reference, Mixpanel’s largest columnar file as of today contains around 2 million events (thanks to the project, user, time sharding). We think the trade-off on time overhead is quite acceptable given the win we get on the unlimited retention window and minimum operational overhead.

Conclusion

In this blog, we discussed our architecture that distributes duplicate identification at the indexing layer and duplicate filtering at the query level. This solution has been live in Mixpanel for the last 6 months.

Brief description:

The Single Source of Truth for Patient Identity and Demographics

- One Patient One Unified Care Record -

Product Goal:

Building a patient-centered digital health ecosystem; Accurate patient identification is a key to achieving the Triple Aim and enables the success of all strategic initiatives. Patient-centric care, population health, accountable care, patient engagement, and value-based reimbursement are just buzzwords without effective patient identity management

An Accurate MPI may be considered the most important resource in a healthcare facility because it’s the link that tracks patient, person, or member activity within an organization (or enterprise) and across patient care settings.

USP: (unique selling proposition)

During the initial assessment, we will load and score your data using the Kodjin Big Data MPI™, which analyzes data quality to identify your real duplication rate.

Then, as part of the overall assessment, you’ll receive a report that outlines a phased-in approach to clean all data in your MPI, and ongoing processes to maintain a 1% (or less) duplication rate.

This report is yours to keep, whether or not you decide to have our team to assist in cleansing your data to remove duplicates, or any of our ongoing MPI maintenance services.

Master Patient Index Software Market:

Introduction

• A master patient index software (MPI) refers to a patient master index, patient registry, and client registry. It is an electronic database system that holds demographic patient information. The benefits of MPI are the elimination of duplicate patient registration entries and the maintenance of a central registry of all patients.

Global Master Patient Index Software Market: Key Trends

• The global master patient index software market is driven by a shift toward paperless data management systems and cost-effective cloud-based technologies for patient data management. Moreover, the adoption of health care data management software has increased for the benefit of health care organizations, medical practitioners, and patient welling and health.
• An increase in adoption of patient record management software and solution for improving patient compliance and effective healthcare management are projected to drive the global master patient index software market during the forecast period
• A master patient index software ensures data integrity with minimum duplicate records. This is projected to drive the global master patient index software market in the next few years.

Global Master Patient Index Software Market Segmentation

• In terms of type, the global master patient index software market can be bifurcated into software and services
• Based on the deployment model, the global master patient index software market can be classified into cloud-based and on-premises. The cloud-based segment is projected to grow at a rapid pace during the forecast period. The adoption of cloud-based technologies will help to simplify the patient data management process.
• In terms of end-user, the global master patient index software market can be divided into healthcare organizations, hospitals, clinics, and others. The healthcare organizations segment accounted for a major share of the global master patient index software market in 2020. This is attributed to increasing in database management software and services for large organizations.

North America accounted for a Major Share of Global Market

• North America is projected to dominate the global master patient index software market during the forecast period due to the presence of advanced healthcare systems and high adoption of advanced technology to maintain patient master index & patient registry
• The majority of market players, research institutes, and academic centers in the region focus on adopting patient data for clinical studies. This factor is anticipated to drive the master patient index software market in North America during the forecast period.

Key Players Operating in Global Master Patient Index Software Market

Major players operating in the global master patient index software market are:

• McKesson Corporation
• Oracle Corporation
• Wipro Limited
• Allscripts Healthcare Solutions, Inc.
• NextGate
• Just Associates, Inc.
• Verato
• QuadraMed Affinity Corporation
• MEDITECH
• ARGO Data Resource Corporation
• Intersystems Corporation
• Epic
• Others

Best practices:

- Just Associates, Inc.:

- NextGate

- QuadraMed Affinity Corporation

- Verato

Global Master Patient Index Software Market: Research Scope

Global Master Patient Index Software Market, by Type

• Software
• Services
• Managed Services

Global Master Patient Index Software Market, by Deployment Mode

• Cloud-based
• On-premises

Global Master Patient Index Software Market, by End-user

• Healthcare Organization
• Hospitals
• Clinics
• Others

Global Master Patient Index Software Market, by Region

• North America
• The U.S.
• Canada
• Europe
• Germany
• France
• The U.K.
• Italy
• Spain
• Rest of Europe
• Asia Pacific
• China
• Japan
• India
• Australia & New Zealand
• Rest of Asia Pacific
• Latin America
• Brazil
• Mexico
• Rest of Latin America
• Middle East & Africa
• GCC Countries
• South Africa
• Rest of the Middle East & Africa

How Does an eMPI Work?

An eMPI constantly looks for potential matches across all data sources that are loaded into the eMPI. The matching is handled by a matching algorithm that locates and assesses the quality of matches even when demographic information is missing or wrong. The algorithm follows the same sort of logic that a well-trained data integrity specialist would when comparing two records to see if they belong to the same person. The algorithm should account for:

• common data entry errors like a transposed month and day in a date of birth misspelled names and inconsistent addresses.
• common false positives like twins or junior/senior
• corrupt records where one person’s information is overlaid with another person’s information.
• default values such as all ones or nines for social security or phone number or ‘baby girl’, or ‘John Doe’ for the name.

When looking for potential matches, an eMPI will start by looking for records that have similar values in key demographic fields. Then it narrows those results by looking at the overall quality of the matches and assigns a score and classification to each potential match.

How Are Matches Classified?

Potential matches are given either a ‘Yes’ or ‘Maybe’ classification. Records that are given a ‘Yes’ classification, meaning that there is no doubt these records belong to the same person, will be automatically linked together by assigning them the same enterprise identifier. No human interaction is required. When two records look like they could be the same person but there are enough differences that it is not safe to auto-link them, they will be given a ‘Maybe’ classification.

Many eMPIs have the ability to use semi-public information from third-party data aggregators like Lexis Nexis to help identify people. You may hear this functionality referred to as Research Automation or Referential Matching. The result is that many of the maybe matches will be turned into yes matches, shrinking the amount of manual work required. Those may be matches that are left become tasks that will need to be reviewed by a data integrity person.

Benefits of MPI:

• Up to date information anytime and anywhere for access to these services
• Ensures medical staff have the correct and most up to date health record for the right patient
• Accurate identification of an individual and their health record quickly when using a health service
• Improves patient safety by reducing the risk of error due to misdirection of clinical information
• Support the provision of cost-effective and timely health care services

General guidelines:

• Medical records in each facility should be uniform and organized systematically
• Authors should be clearly identifiable
• All entries should be permanent
• Abbreviations should not be used
• Errors should include a strikeout with a single line and initiate with a date

There are four objectives that a modern eMPI should support. They are:

1. Identifying that records from different source systems belong to the same patient and linking them together even when key information such as social security number is missing or wrong. To link the records together, an eMPI will assign the same enterprise identifier to each record belonging to the same patient in each source system.
2. Making use of outside semi-public data to reduce or eliminate the manual effort involved in patient matching.
3. Generating a golden record that contains the best-known information for a person whose information is spread across many different sources.
4. “De-duping” a transactional system by identifying cases where there’s more than one record for the same patient and providing workflow for resolving them.

У меня немного траблы с технической частью, из-за этого хромает понимание процесса в целом.

Определение: я ПО/БА в компании которая развивает новый протокол обмена мед данными для "импортных клиентов". Эта штука (FHIR HL7) сильно урегулированна, все строится на так званных ресурсах (разные профайлы для вендоров, страховых и клиента) эти делятся еще на 100тыс профайлов и тд. (фракталы)

Команда супер! Хочу с твоей помощью лучше разобраться, чтоб главное – научиться ставить граммотные вопросы BA, PO, DevOps, SA (system analytic/admin)

Пример;

Фалы хранятся и обмениваются в XLM/JSON через https протокол (микросерверы rest api)

Начало:

Не пойму куда бежать, тк со мной никто не нянчится, все заняты :)дали 2 недели на чтоб вникнуть + 3 мес пробный, чтоб выкатить детальный Беклог под MPI (мастер индекс по пациенту) это новое направление в компании, хотят мое видение

Обязанности: 

1. Формулировка целей проекта
2. Управление бэклогом продукта + депенденсис (зависимости)
3. Визия по разработке продукта
4. Обеспечение прозрачности + постоянный фидбек (обратная связь)
5. Общение со стейкхолдерами (заинтересованными сторонами)
6. + Написание продакт бэклога + сопутствующей документации (acceptance criteria, DoD) +
• Define goal and roadmap
• Responsibilities
• Tangible tasks

1. PHIR umbrella (micro services)
2. Understand conditions and classes
3. Practical use cases + resources + theory
4. Design first approach/deliverables; can I take a look at how API, data model, architecture, sigma diagrams, implementation notes look like
5. Take a look at customer-centric claims (not general) + sketches

1. IDEAL role of PO. Какие скилы необходимы для отличного ПМ (project manager/product owner)??? твое видение/опыт

Софт - 5 мин

Хардскилс - 5 мин

Знания домена - 5 мин

На что сделать главный акцент?

А. ФАЙЕР (это то что мы имплементируем) http://hl7.org/fhir/ я читаю доку, но толку почему-то мало((: еще сделал таблицу в экселе https://docs.google.com/spreadsheets/d/1rgDlyreLAy68A9usblNwZnln7P7UNvKpj3CCVi1muwY/edit?usp=sharing

Б. У нас скрам + аджайл, спринты 1 мес, утром дейли митинги, понимать прод цикл CI/CD + internal/ external server + stack

2. Какая зона ответственности у PO? По приоритетам:

• 1
• 2
• 3
• 4

3. №1 Confluence

Есть 4 основных спейса:

На что сделать акцент чтобы в будущем понимать и рисовать логические UML схемы? Как правильно задавать вопросы SA?

Пример:

Как ты справляешься с большим количеством абривиатуры? Нужно ли ПМу запоминать команды например - месседж тайпс (ниже)?

Есть ряд вопросов по Постману работа с :

Хедер

Бади

Метаданные

Есть файлы которые можно потрогать. Using the Collection Runner

Есть вопросы по CLI и Config management; я так и не понял, они хотят чтоб у провайдера обмен был сразу через кофиг менеджемент #Bash

Я закончил пару курсов по Https, rest api. Мне нужно выговориться...

Вопросы по Виртуализации: использование Докера

РАБОТА С GitHub, GitLab компания предоставила доступ

Есть ряд вопросов (нужно разшарить экран)

Не пойму нужно ли мне начинать программировать

Вот их рабочий стек:

Хочу понять стек лучше, чтоб не задавать глупых вопросов.

Жду обратную от тебя связь. Заранее спасибо 🤝