US10878001B2 - Identifying relationships among a group of indicators - Google Patents

Identifying relationships among a group of indicators Download PDF

Info

Publication number
US10878001B2
US10878001B2 US16/019,656 US201816019656A US10878001B2 US 10878001 B2 US10878001 B2 US 10878001B2 US 201816019656 A US201816019656 A US 201816019656A US 10878001 B2 US10878001 B2 US 10878001B2
Authority
US
United States
Prior art keywords
indicators
score
group
correlation score
fitting function
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Expired - Fee Related, expires
Application number
US16/019,656
Other versions
US20200004871A1 (en
Inventor
Al Chakra
Tian Ming Pan
Peng Fei Tian
Chu Yun Cloud Tony
Cheng Fang Wang
RenFu Ma
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
International Business Machines Corp
Original Assignee
International Business Machines Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by International Business Machines Corp filed Critical International Business Machines Corp
Priority to US16/019,656 priority Critical patent/US10878001B2/en
Assigned to INTERNATIONAL BUSINESS MACHINES CORPORATION reassignment INTERNATIONAL BUSINESS MACHINES CORPORATION ASSIGNMENT OF ASSIGNORS INTEREST (SEE DOCUMENT FOR DETAILS). Assignors: MA, RENFU, PAN, TIAN MING, TIAN, PENG FEI, TONY, CHU YUN CLOUD, WANG, CHENG FANG, CHAKRA, AL
Publication of US20200004871A1 publication Critical patent/US20200004871A1/en
Application granted granted Critical
Publication of US10878001B2 publication Critical patent/US10878001B2/en
Expired - Fee Related legal-status Critical Current
Adjusted expiration legal-status Critical

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/28Databases characterised by their database models, e.g. relational or object models
    • G06F16/284Relational databases
    • G06F16/285Clustering or classification
    • G06F16/287Visualization; Browsing
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N5/00Computing arrangements using knowledge-based models
    • G06N5/02Knowledge representation; Symbolic representation
    • G06N5/022Knowledge engineering; Knowledge acquisition
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/26Visual data mining; Browsing structured data
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10Complex mathematical operations
    • G06F17/15Correlation function computation including computation of convolution operations
    • GPHYSICS
    • G06COMPUTING; CALCULATING OR COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F17/00Digital computing or data processing equipment or methods, specially adapted for specific functions
    • G06F17/10Complex mathematical operations
    • G06F17/18Complex mathematical operations for evaluating statistical data, e.g. average values, frequency distributions, probability functions, regression analysis

Definitions

  • the invention relates generally to identifying relationships among a group of indicators and, more specifically, to analyzing pairs of indicators to identify relationships between indicators.
  • Computing systems such as transaction processing systems, are a source of data that can be used to gain valuable business insights. Accordingly, research has been performed on using machine learning to identify these business insights. In most cases, when user trains a machine learning model for each indicator in a computing system, the indicators are evaluated as a time-series of data points.
  • a system for identifying relationships among a group of indicators includes a memory having computer readable computer instructions, and a processor for executing the computer readable instructions.
  • the computer readable instructions include instructions for obtaining the group of indicators, including a number of indicators relating to an operation of a computing system.
  • the computer readable instructions also include instructions for creating pairs of indicators, wherein the pairs of indicators include all possible combination of the group of indicators.
  • the computer readable instructions further include instructions for, for each pair of indicators, calculating a linear correlation score, calculating a fitting function score, determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score and storing the final correlation score in a relationship database.
  • the computer readable instructions also include instructions for creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators.
  • a method for identifying relationships among a group of indicators includes obtaining the group of indicators, including a number of indicators relating to an operation of a computing system. The method also includes creating pairs of indicators, wherein the pairs of indicators includes all possible combination of the group of indicators. The method further includes, for each pair of indicators, calculating a linear correlation score, calculating a fitting function score, determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score and storing the final correlation score in a relationship database. The method also includes creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators.
  • a computer program product includes a computer readable storage medium having program instructions embodied therewith.
  • the computer readable storage medium is not a transitory signal per se.
  • the program instructions are executable by a computer processor to cause the computer processor to perform a method.
  • the method includes obtaining the group of indicators, including a number of indicators relating to an operation of a computing system.
  • the method also includes creating pairs of indicators, wherein the pairs of indicators includes all possible combination of the group of indicators.
  • the method further includes, for each pair of indicators, calculating a linear correlation score, calculating a fitting function score, determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score and storing the final correlation score in a relationship database.
  • the method also includes creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators.
  • FIG. 1 depicts an exemplary computer system capable of implementing one or more embodiments of the present invention
  • FIG. 2 depicts a system for identifying relationships among a group of indicators according to one or more embodiments of the present invention
  • FIG. 3 depicts a flow diagram of a method for identifying relationships among a group of indicators according to one or more embodiments of the present invention
  • FIG. 4 depicts a ranking map for a selected indicator from a group of indicators according to one or more embodiments of the present invention.
  • FIG. 5 depicts a correlation map for a group of indicators according to one or more embodiments of the present invention.
  • compositions comprising, “comprising,” “includes,” “including,” “has,” “having,” “contains” or “containing,” or any other variation thereof, are intended to cover a non-exclusive inclusion.
  • a composition, a mixture, process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but can include other elements not expressly listed or inherent to such composition, mixture, process, method, article, or apparatus.
  • exemplary is used herein to mean “serving as an example, instance or illustration.” Any embodiment or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or designs.
  • the terms “at least one” and “one or more” may be understood to include any integer number greater than or equal to one, i.e. one, two, three, four, etc.
  • the terms “a plurality” may be understood to include any integer number greater than or equal to two, i.e. two, three, four, five, etc.
  • connection may include both an indirect “connection” and a direct “connection.”
  • FIG. 1 illustrates a high-level block diagram showing an example of a computer-based system 100 useful for implementing one or more embodiments of the invention.
  • computer system 100 includes a communication path 126 , which connects computer system 100 to additional systems and may include one or more wide area networks (WANs) and/or local area networks (LANs) such as the internet, intranet(s), and/or wireless communication network(s).
  • WANs wide area networks
  • LANs local area networks
  • Computer system 100 and additional systems are in communication via communication path 126 , (e.g., to communicate data between them).
  • Computer system 100 includes one or more processors, such as processor 102 .
  • Processor 102 is connected to a communication infrastructure 104 (e.g., a communications bus, cross-over bar, or network).
  • Computer system 100 can include a display interface 106 that forwards graphics, text, and other data from communication infrastructure 104 (or from a frame buffer not shown) for display on a display unit 108 .
  • Computer system 100 also includes a main memory 110 , preferably random access memory (RAM), and may also include a secondary memory 112 .
  • Secondary memory 112 may include, for example, a hard disk drive 114 and/or a removable storage drive 116 , representing, for example, a floppy disk drive, a magnetic tape drive, or an optical disk drive.
  • Removable storage drive 116 reads from and/or writes to a removable storage unit 118 in a manner well known to those having ordinary skill in the art.
  • Removable storage unit 118 represents, for example, a floppy disk, a compact disc, a magnetic tape, or an optical disk, etc. which is read by and written to by a removable storage drive 116 .
  • removable storage unit 118 includes a computer readable medium having stored therein computer software and/or data.
  • secondary memory 112 may include other similar means for allowing computer programs or other instructions to be loaded into the computer system.
  • Such means may include, for example, a removable storage unit 120 and an interface 122 .
  • Examples of such means may include a program package and package interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, and other removable storage units 120 and interfaces 122 which allow software and data to be transferred from the removable storage unit 120 to computer system 100 .
  • Computer system 100 may also include a communications interface 124 .
  • Communications interface 124 allows software and data to be transferred between the computer system and external devices. Examples of communications interface 124 may include a modem, a network interface (such as an Ethernet card), a communications port, or a PCM-CIA slot and card, etc.
  • Software and data transferred via communications interface 124 are in the form of signals which may be, for example, electronic, electromagnetic, optical, or other signals capable of being received by communications interface 124 . These signals are provided to communications interface 124 via communication path (i.e., channel) 126 .
  • Communication path 126 carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and/or other communications channels.
  • computer program medium In the present disclosure, the terms “computer program medium,” “computer usable medium,” and “computer readable medium” are used to generally refer to media such as main memory 110 and secondary memory 112 , removable storage drive 116 , and a hard disk installed in hard disk drive 114 .
  • Computer programs also called computer control logic
  • main memory 110 main memory 110
  • secondary memory 112 Computer programs may also be received via communications interface 124 .
  • Such computer programs when run, enable the computer system to perform the features of the present disclosure as discussed herein.
  • the computer programs when run, enable processor 102 to perform the features of the computer system. Accordingly, such computer programs represent controllers of the computer system.
  • a system for identifying relationships among a group of n indicators includes performance data, or metrics, for a separate computing system, such as a transaction processing system and can also include discretized time data.
  • This group of indicators are split into n(n ⁇ 1) pairs of indicators and each pair of indicators is analyzed and assigned a linear value score and a fitting function score. If either the absolute value of linear value score or one minus the fitting function score exceed a threshold value, the larger of the absolute value of linear value score and one minus the fitting function score is stored as a final correlation score in a relationship database.
  • the relationship database stores the final correlation score for each combination of indicators and optionally the function associated with that final correlation score.
  • the data stored in the relationship database can be visualized using a ranking map that illustrates a group of indicators that are correlated with a user selected indicator or using a correlation map that includes a node for each indicator and a connector between each related pair of indicators.
  • the thickness of a connector between the nodes of the ranking or correlation map is positively correlated with the final correlation score associated with the pair of indicators associated with the nodes.
  • the system 200 shown in FIG. 2 includes a computing system 230 in communication with a processing system 210 via a communications network 220 .
  • the communications network 220 may be one or more of, or a combination of, public (e.g., Internet), private (e.g., local area network, wide area network, virtual private network), and may include wireless and wireline transmission systems (e.g., satellite, cellular network, terrestrial networks, etc.).
  • the performance of the computing system 230 is characterized by a plurality of performance indicators 232 .
  • the computing system 230 is a transaction processing system and the performance indicators include a transaction processing rate, a total number of pending transactions, an average transaction response time, and the like.
  • the performance indicators 232 are collected by the computing system 230 and are provided to the processing system 210 for analysis.
  • the processing system 210 includes a linear correlation scoring module 211 , a fitting function training module 212 , a fitting function scoring module 213 , a combined scoring module 214 , and a relationship database 215 .
  • the processing system 210 may be implemented as a computer system such as the one shown in FIG. 1 .
  • the processing system 210 receives the performance indicators 232 from the computing system 230 .
  • the performance indicators 232 include a plurality of indicators that are combined with a time indicator to form a group of n indicators.
  • the processing system 210 performs a standardization on the performance indicators 232 received so that the data can be processed by the linear correlation scoring module 211 , a fitting function training module 212 , a fitting function scoring module 213 .
  • one performance indicator 232 may be a performance metric that is captured once every ten seconds while another performance indicator 232 is captured once every five seconds.
  • the standardization can include reducing the data set from the more frequently captured data by discarding every other data point.
  • the standardization can include interpolating or duplicating data in the less frequently captured data set. Additional data standardization techniques can also be performed.
  • the computing system 230 is configured to provide the processing system 210 with data, such as the sampling rate, for each of the performance indicators 232 .
  • the processing system 210 is configured to create n(n ⁇ 1) groups from the n indicators, which represent every possible combination of the indicators. Once the pairs have been created, the linear correlation scoring module 211 calculates a linear correlation score for each pair. In one embodiment, the linear correlation score is calculated by:
  • ⁇ xy ⁇ ( X i - X _ ) ⁇ ( Y i - Y _ ) ⁇ ( X i - X _ ) 2 ⁇ ⁇ ( Y i ⁇ - Y _ ) 2
  • fitting function training module 212 can use any of a variety of known techniques to analyze the data pair and to responsively identify a function that represents the correlation between the indicators.
  • the function scoring module 213 is provided to the function scoring module 213 along with the indicator data.
  • the function scoring module 213 then calculates a fitting function score that indicates how well the pair of indicators conforms to the identified function.
  • the fitting function score can be a value of one minus a root mean squared deviation of:
  • h is the identified function
  • X is a matrix containing all the feature values (excluding labels) of all instances in the dataset.
  • i row is equal to the transpose of x (i) .
  • the combined scoring module 214 evaluates an absolute value of the linear correlation score and one minus the fitting function score to determine a final correlation score. Since a higher root mean square indicates a poorer fit between the data and the identified function, one minus the fitting function score is used to by the combined scoring module 214 .
  • the final correlation score is determined to be zero based on determining that an absolute value of the linear correlation score is less than a threshold number, for example 0.5, and that one minus the fitting function score is less than the threshold number. Otherwise, the final correlation score is determined to be a greater of the absolute value of the linear correlation score and one minus the fitting function score.
  • the final correlation score is stored in the relationship database 215 .
  • the function that corresponds to the calculated final correlation score is also stored in the relationship database 215 .
  • the final correlation score is the linear correlation score
  • a function identified by the fitting function training module 212 is stored in the relationship database 215 .
  • the method 300 includes obtaining the group of indicators, including a number of indicators relating to the operation of a computing system, as shown at block 302 .
  • the group of indicators includes a time indicator that has been discretized.
  • the method 300 includes creating pairs of indicators, wherein the pairs of indicators include all possible combination of the group of indicators.
  • the method 300 further includes calculating a linear correlation score and a fitting function score for each pair of indicators, as shown at block 306 .
  • the method 300 includes determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score.
  • the final correlation score in stored a relationship database, as shown at block 310 .
  • the method 300 includes creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength of the relationship among the group of indicators.
  • the graphical display can include a ranking map 400 , such as the one shown in FIG. 4 , or a correlation map 500 , such as the one shown in FIG. 5 .
  • the ranking map 400 is created based on a selected indicator 402 that is identified by a user and includes nodes 404 for all of the indicators that have a non-zero final correlation score with the selected indicator 402 .
  • a thickness of the connections 403 , 405 between the selected indicator 402 and the nodes 404 are used to illustrate the strength of the correlation between the selected indicator 402 and the indicator represented by the nodes 404 .
  • more highly correlated indicators are connected to the selected indicator 402 by thicker lines.
  • a size of the nodes 404 and the selected indicator 402 can be based on a numeric value for indicator associated with the node.
  • FIG. 5 depicts a correlation map 500 for a group of indicators according to one or more embodiments of the present invention.
  • the topology of the correlation map 500 is similar to the ranking map 400 , but the correlation map 500 is configured to illustrate all of the data stored in the relationship database.
  • the ranking map and/or the correlation map can be used to visualize the relationships between indicators and to evaluate the effects of changes to a desired indicator.
  • the ranking map 400 can display a value for each of the nodes 404 and the selected indicator 402 and can be configured to allow the user to propose a change to one of these values.
  • the ranking map can calculate new values for each of the displayed nodes that illustrate changes that would be needed to be made to the computing system to effectuate the proposed change.
  • the ranking map 400 may include a selected indicator 402 of an average response time and correlated indicators of a number of processing cores, memory utilization, transactions rate, and the like. If the user were to enter a proposed average response time that was half of the displayed average response time, the ranking map 400 would be updated to illustrate new values for the number of processing cores, memory utilization, transactions rate, and the like that would be needed to achieve the desired reduction in the response time. These values are calculated based on the final correlation score and functions stored in the relationship database.
  • the present invention may be a system, a method, and/or a computer program product.
  • the computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.
  • the computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device.
  • the computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing.
  • a non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing.
  • RAM random access memory
  • ROM read-only memory
  • EPROM or Flash memory erasable programmable read-only memory
  • SRAM static random access memory
  • CD-ROM compact disc read-only memory
  • DVD digital versatile disk
  • memory stick a floppy disk
  • a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon
  • a computer readable storage medium is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
  • Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network.
  • the network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers.
  • a network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
  • Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages.
  • the computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server.
  • the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider).
  • electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.
  • These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
  • the computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
  • each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s).
  • the functions noted in the block may occur out of the order noted in the figures.
  • two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved.

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Computational Mathematics (AREA)
  • Mathematical Analysis (AREA)
  • Mathematical Optimization (AREA)
  • Pure & Applied Mathematics (AREA)
  • Software Systems (AREA)
  • Algebra (AREA)
  • Computing Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computational Linguistics (AREA)
  • Evolutionary Computation (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Operations Research (AREA)
  • Probability & Statistics with Applications (AREA)
  • Information Retrieval, Db Structures And Fs Structures Therefor (AREA)

Abstract

Methods, systems, and computer program products for identifying relationships among a group of indicators are provided. Aspects include obtaining the group of indicators, including a number of indicators relating to an operation of a computing system. Aspects also include creating pairs of indicators, wherein the pairs of indicators includes all possible combination of the group of indicators. Aspects further include, for each pair of indicators, calculating a linear correlation score, calculating a fitting function score, determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score and storing the final correlation score in a relationship database. Aspects also include creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators.

Description

BACKGROUND
The invention relates generally to identifying relationships among a group of indicators and, more specifically, to analyzing pairs of indicators to identify relationships between indicators.
Computing systems, such as transaction processing systems, are a source of data that can be used to gain valuable business insights. Accordingly, research has been performed on using machine learning to identify these business insights. In most cases, when user trains a machine learning model for each indicator in a computing system, the indicators are evaluated as a time-series of data points.
SUMMARY
According to an embodiment, a system for identifying relationships among a group of indicators is provided. The system includes a memory having computer readable computer instructions, and a processor for executing the computer readable instructions. The computer readable instructions include instructions for obtaining the group of indicators, including a number of indicators relating to an operation of a computing system. The computer readable instructions also include instructions for creating pairs of indicators, wherein the pairs of indicators include all possible combination of the group of indicators. The computer readable instructions further include instructions for, for each pair of indicators, calculating a linear correlation score, calculating a fitting function score, determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score and storing the final correlation score in a relationship database. The computer readable instructions also include instructions for creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators.
According to another embodiment, a method for identifying relationships among a group of indicators is provided. The method includes obtaining the group of indicators, including a number of indicators relating to an operation of a computing system. The method also includes creating pairs of indicators, wherein the pairs of indicators includes all possible combination of the group of indicators. The method further includes, for each pair of indicators, calculating a linear correlation score, calculating a fitting function score, determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score and storing the final correlation score in a relationship database. The method also includes creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators.
According to a further embodiment, a computer program product is provided. The computer program product includes a computer readable storage medium having program instructions embodied therewith. The computer readable storage medium is not a transitory signal per se. The program instructions are executable by a computer processor to cause the computer processor to perform a method. The method includes obtaining the group of indicators, including a number of indicators relating to an operation of a computing system. The method also includes creating pairs of indicators, wherein the pairs of indicators includes all possible combination of the group of indicators. The method further includes, for each pair of indicators, calculating a linear correlation score, calculating a fitting function score, determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score and storing the final correlation score in a relationship database. The method also includes creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators.
Additional features and advantages are realized through the techniques of the invention. Other embodiments and aspects of the invention are described in detail herein and are considered a part of the claimed invention. For a better understanding of the invention with the advantages and the features, refer to the description and to the drawings.
BRIEF DESCRIPTION OF THE DRAWINGS
The subject matter which is regarded as the invention is particularly pointed out and distinctly claimed in the claims at the conclusion of the specification. The forgoing and other features, and advantages of the invention are apparent from the following detailed description taken in conjunction with the accompanying drawings, in which:
FIG. 1 depicts an exemplary computer system capable of implementing one or more embodiments of the present invention;
FIG. 2 depicts a system for identifying relationships among a group of indicators according to one or more embodiments of the present invention;
FIG. 3 depicts a flow diagram of a method for identifying relationships among a group of indicators according to one or more embodiments of the present invention;
FIG. 4 depicts a ranking map for a selected indicator from a group of indicators according to one or more embodiments of the present invention; and
FIG. 5 depicts a correlation map for a group of indicators according to one or more embodiments of the present invention.
DETAILED DESCRIPTION
Various embodiments of the invention are described herein with reference to the related drawings. Alternative embodiments of the invention can be devised without departing from the scope of this invention. Various connections and positional relationships (e.g., over, below, adjacent, etc.) are set forth between elements in the following description and in the drawings. These connections and/or positional relationships, unless specified otherwise, can be direct or indirect, and the present invention is not intended to be limiting in this respect. Accordingly, a coupling of entities can refer to either a direct or an indirect coupling, and a positional relationship between entities can be a direct or indirect positional relationship. Moreover, the various tasks and process steps described herein can be incorporated into a more comprehensive procedure or process having additional steps or functionality not described in detail herein.
The following definitions and abbreviations are to be used for the interpretation of the claims and the specification. As used herein, the terms “comprises,” “comprising,” “includes,” “including,” “has,” “having,” “contains” or “containing,” or any other variation thereof, are intended to cover a non-exclusive inclusion. For example, a composition, a mixture, process, method, article, or apparatus that comprises a list of elements is not necessarily limited to only those elements but can include other elements not expressly listed or inherent to such composition, mixture, process, method, article, or apparatus.
Additionally, the term “exemplary” is used herein to mean “serving as an example, instance or illustration.” Any embodiment or design described herein as “exemplary” is not necessarily to be construed as preferred or advantageous over other embodiments or designs. The terms “at least one” and “one or more” may be understood to include any integer number greater than or equal to one, i.e. one, two, three, four, etc. The terms “a plurality” may be understood to include any integer number greater than or equal to two, i.e. two, three, four, five, etc. The term “connection” may include both an indirect “connection” and a direct “connection.”
The terms “about,” “substantially,” “approximately,” and variations thereof, are intended to include the degree of error associated with measurement of the particular quantity based upon the equipment available at the time of filing the application. For example, “about” can include a range of ±8% or 5%, or 2% of a given value.
For the sake of brevity, conventional techniques related to making and using aspects of the invention may or may not be described in detail herein. In particular, various aspects of computing systems and specific computer programs to implement the various technical features described herein are well known. Accordingly, in the interest of brevity, many conventional implementation details are only mentioned briefly herein or are omitted entirely without providing the well-known system and/or process details.
Turning now to a more detailed description of aspects of the present invention, FIG. 1 illustrates a high-level block diagram showing an example of a computer-based system 100 useful for implementing one or more embodiments of the invention. Although one exemplary computer system 100 is shown, computer system 100 includes a communication path 126, which connects computer system 100 to additional systems and may include one or more wide area networks (WANs) and/or local area networks (LANs) such as the internet, intranet(s), and/or wireless communication network(s). Computer system 100 and additional systems are in communication via communication path 126, (e.g., to communicate data between them).
Computer system 100 includes one or more processors, such as processor 102. Processor 102 is connected to a communication infrastructure 104 (e.g., a communications bus, cross-over bar, or network). Computer system 100 can include a display interface 106 that forwards graphics, text, and other data from communication infrastructure 104 (or from a frame buffer not shown) for display on a display unit 108. Computer system 100 also includes a main memory 110, preferably random access memory (RAM), and may also include a secondary memory 112. Secondary memory 112 may include, for example, a hard disk drive 114 and/or a removable storage drive 116, representing, for example, a floppy disk drive, a magnetic tape drive, or an optical disk drive. Removable storage drive 116 reads from and/or writes to a removable storage unit 118 in a manner well known to those having ordinary skill in the art. Removable storage unit 118 represents, for example, a floppy disk, a compact disc, a magnetic tape, or an optical disk, etc. which is read by and written to by a removable storage drive 116. As will be appreciated, removable storage unit 118 includes a computer readable medium having stored therein computer software and/or data.
In some alternative embodiments of the invention, secondary memory 112 may include other similar means for allowing computer programs or other instructions to be loaded into the computer system. Such means may include, for example, a removable storage unit 120 and an interface 122. Examples of such means may include a program package and package interface (such as that found in video game devices), a removable memory chip (such as an EPROM or PROM) and associated socket, and other removable storage units 120 and interfaces 122 which allow software and data to be transferred from the removable storage unit 120 to computer system 100.
Computer system 100 may also include a communications interface 124. Communications interface 124 allows software and data to be transferred between the computer system and external devices. Examples of communications interface 124 may include a modem, a network interface (such as an Ethernet card), a communications port, or a PCM-CIA slot and card, etc. Software and data transferred via communications interface 124 are in the form of signals which may be, for example, electronic, electromagnetic, optical, or other signals capable of being received by communications interface 124. These signals are provided to communications interface 124 via communication path (i.e., channel) 126. Communication path 126 carries signals and may be implemented using wire or cable, fiber optics, a phone line, a cellular phone link, an RF link, and/or other communications channels.
In the present disclosure, the terms “computer program medium,” “computer usable medium,” and “computer readable medium” are used to generally refer to media such as main memory 110 and secondary memory 112, removable storage drive 116, and a hard disk installed in hard disk drive 114. Computer programs (also called computer control logic) are stored in main memory 110, and/or secondary memory 112. Computer programs may also be received via communications interface 124. Such computer programs, when run, enable the computer system to perform the features of the present disclosure as discussed herein. In particular, the computer programs, when run, enable processor 102 to perform the features of the computer system. Accordingly, such computer programs represent controllers of the computer system.
In exemplary embodiments, a system for identifying relationships among a group of n indicators is provided. The group of indicators include performance data, or metrics, for a separate computing system, such as a transaction processing system and can also include discretized time data. This group of indicators are split into n(n−1) pairs of indicators and each pair of indicators is analyzed and assigned a linear value score and a fitting function score. If either the absolute value of linear value score or one minus the fitting function score exceed a threshold value, the larger of the absolute value of linear value score and one minus the fitting function score is stored as a final correlation score in a relationship database. In exemplary embodiments, the relationship database stores the final correlation score for each combination of indicators and optionally the function associated with that final correlation score. The data stored in the relationship database can be visualized using a ranking map that illustrates a group of indicators that are correlated with a user selected indicator or using a correlation map that includes a node for each indicator and a connector between each related pair of indicators. In exemplary embodiments, the thickness of a connector between the nodes of the ranking or correlation map is positively correlated with the final correlation score associated with the pair of indicators associated with the nodes.
Turning now to FIG. 2, a system 200 upon which identifying relationships among a group of indicators may be implemented will now be described in accordance with an embodiment. The system 200 shown in FIG. 2 includes a computing system 230 in communication with a processing system 210 via a communications network 220. The communications network 220 may be one or more of, or a combination of, public (e.g., Internet), private (e.g., local area network, wide area network, virtual private network), and may include wireless and wireline transmission systems (e.g., satellite, cellular network, terrestrial networks, etc.).
The performance of the computing system 230 is characterized by a plurality of performance indicators 232. In one embodiment, the computing system 230 is a transaction processing system and the performance indicators include a transaction processing rate, a total number of pending transactions, an average transaction response time, and the like. The performance indicators 232 are collected by the computing system 230 and are provided to the processing system 210 for analysis.
The processing system 210 includes a linear correlation scoring module 211, a fitting function training module 212, a fitting function scoring module 213, a combined scoring module 214, and a relationship database 215. The processing system 210 may be implemented as a computer system such as the one shown in FIG. 1. In exemplary embodiments, the processing system 210 receives the performance indicators 232 from the computing system 230. The performance indicators 232 include a plurality of indicators that are combined with a time indicator to form a group of n indicators.
In exemplary embodiments, the processing system 210 performs a standardization on the performance indicators 232 received so that the data can be processed by the linear correlation scoring module 211, a fitting function training module 212, a fitting function scoring module 213. For example, one performance indicator 232 may be a performance metric that is captured once every ten seconds while another performance indicator 232 is captured once every five seconds. In this example, the standardization can include reducing the data set from the more frequently captured data by discarding every other data point. Likewise, the standardization can include interpolating or duplicating data in the less frequently captured data set. Additional data standardization techniques can also be performed. In exemplary embodiments, the computing system 230 is configured to provide the processing system 210 with data, such as the sampling rate, for each of the performance indicators 232.
The processing system 210 is configured to create n(n−1) groups from the n indicators, which represent every possible combination of the indicators. Once the pairs have been created, the linear correlation scoring module 211 calculates a linear correlation score for each pair. In one embodiment, the linear correlation score is calculated by:
ρ xy = ( X i - X _ ) ( Y i - Y _ ) ( X i - X _ ) 2 ( Y i - Y _ ) 2
Each pair is also provided to the fitting function training module 212, which can use any of a variety of known techniques to analyze the data pair and to responsively identify a function that represents the correlation between the indicators. Once the function is identified, it is provided to the function scoring module 213 along with the indicator data. The function scoring module 213 then calculates a fitting function score that indicates how well the pair of indicators conforms to the identified function. In one embodiment, the fitting function score can be a value of one minus a root mean squared deviation of:
RMSE ( X , h ) = 1 m i = 1 m ( h ( x ( i ) ) - y ( i ) ) 2
where h is the identified function and X is a matrix containing all the feature values (excluding labels) of all instances in the dataset. There is one row per instance and the i row is equal to the transpose of x(i). m is the number of instances in the dataset you are measuring the RMSE of. For example, if you are evaluating the RMSE on a validation set of 2,000 districts, then m=2,000.
Next, the combined scoring module 214 evaluates an absolute value of the linear correlation score and one minus the fitting function score to determine a final correlation score. Since a higher root mean square indicates a poorer fit between the data and the identified function, one minus the fitting function score is used to by the combined scoring module 214. In exemplary embodiments, the final correlation score is determined to be zero based on determining that an absolute value of the linear correlation score is less than a threshold number, for example 0.5, and that one minus the fitting function score is less than the threshold number. Otherwise, the final correlation score is determined to be a greater of the absolute value of the linear correlation score and one minus the fitting function score.
The final correlation score is stored in the relationship database 215. In exemplary embodiments, the function that corresponds to the calculated final correlation score is also stored in the relationship database 215. For example, if the final correlation score is the linear correlation score, a liner function in the form of y=mx+b is stored in the relationship database 215. If the final correlation score is the fitting function score then a function identified by the fitting function training module 212 is stored in the relationship database 215.
Referring now to FIG. 3 a flow diagram of a method for identifying relationships among a group of indicators according to one or more embodiments of the present invention is shown. The method 300 includes obtaining the group of indicators, including a number of indicators relating to the operation of a computing system, as shown at block 302. In exemplary embodiments, the group of indicators includes a time indicator that has been discretized. Next, as shown at block 304, the method 300 includes creating pairs of indicators, wherein the pairs of indicators include all possible combination of the group of indicators. The method 300 further includes calculating a linear correlation score and a fitting function score for each pair of indicators, as shown at block 306. Next, as shown at block 308, the method 300 includes determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score. The final correlation score in stored a relationship database, as shown at block 310. Next, as shown at block 312, the method 300 includes creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength of the relationship among the group of indicators. In exemplary embodiments, the graphical display can include a ranking map 400, such as the one shown in FIG. 4, or a correlation map 500, such as the one shown in FIG. 5.
Referring now to FIG. 4 a ranking map 400 for a selected indicator 402 from a group of indicators according to one or more embodiments of the present invention is shown. The ranking map 400 is created based on a selected indicator 402 that is identified by a user and includes nodes 404 for all of the indicators that have a non-zero final correlation score with the selected indicator 402. A thickness of the connections 403, 405 between the selected indicator 402 and the nodes 404 are used to illustrate the strength of the correlation between the selected indicator 402 and the indicator represented by the nodes 404. For example, more highly correlated indicators are connected to the selected indicator 402 by thicker lines. In addition, a size of the nodes 404 and the selected indicator 402 can be based on a numeric value for indicator associated with the node.
FIG. 5 depicts a correlation map 500 for a group of indicators according to one or more embodiments of the present invention. The topology of the correlation map 500 is similar to the ranking map 400, but the correlation map 500 is configured to illustrate all of the data stored in the relationship database.
In exemplary embodiments, the ranking map and/or the correlation map can be used to visualize the relationships between indicators and to evaluate the effects of changes to a desired indicator. The ranking map 400 can display a value for each of the nodes 404 and the selected indicator 402 and can be configured to allow the user to propose a change to one of these values. In response to receiving the proposed change to one of these values, the ranking map can calculate new values for each of the displayed nodes that illustrate changes that would be needed to be made to the computing system to effectuate the proposed change.
For example, the ranking map 400 may include a selected indicator 402 of an average response time and correlated indicators of a number of processing cores, memory utilization, transactions rate, and the like. If the user were to enter a proposed average response time that was half of the displayed average response time, the ranking map 400 would be updated to illustrate new values for the number of processing cores, memory utilization, transactions rate, and the like that would be needed to achieve the desired reduction in the response time. These values are calculated based on the final correlation score and functions stored in the relationship database.
The present invention may be a system, a method, and/or a computer program product. The computer program product may include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present invention.
The computer readable storage medium can be a tangible device that can retain and store instructions for use by an instruction execution device. The computer readable storage medium may be, for example, but is not limited to, an electronic storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. A non-exhaustive list of more specific examples of the computer readable storage medium includes the following: a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), a static random access memory (SRAM), a portable compact disc read-only memory (CD-ROM), a digital versatile disk (DVD), a memory stick, a floppy disk, a mechanically encoded device such as punch-cards or raised structures in a groove having instructions recorded thereon, and any suitable combination of the foregoing. A computer readable storage medium, as used herein, is not to be construed as being transitory signals per se, such as radio waves or other freely propagating electromagnetic waves, electromagnetic waves propagating through a waveguide or other transmission media (e.g., light pulses passing through a fiber-optic cable), or electrical signals transmitted through a wire.
Computer readable program instructions described herein can be downloaded to respective computing/processing devices from a computer readable storage medium or to an external computer or external storage device via a network, for example, the Internet, a local area network, a wide area network and/or a wireless network. The network may comprise copper transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium within the respective computing/processing device.
Computer readable program instructions for carrying out operations of the present invention may be assembler instructions, instruction-set-architecture (ISA) instructions, machine instructions, machine dependent instructions, microcode, firmware instructions, state-setting data, or either source code or object code written in any combination of one or more programming languages, including an object oriented programming language such as Smalltalk, C++ or the like, and conventional procedural programming languages, such as the “C” programming language or similar programming languages. The computer readable program instructions may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (for example, through the Internet using an Internet Service Provider). In some embodiments, electronic circuitry including, for example, programmable logic circuitry, field-programmable gate arrays (FPGA), or programmable logic arrays (PLA) may execute the computer readable program instructions by utilizing state information of the computer readable program instructions to personalize the electronic circuitry, in order to perform aspects of the present invention.
Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer readable program instructions.
These computer readable program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions/acts specified in the flowchart and/or block diagram block or blocks. These computer readable program instructions may also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and/or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including instructions which implement aspects of the function/act specified in the flowchart and/or block diagram block or blocks.
The computer readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions/acts specified in the flowchart and/or block diagram block or blocks.
The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and/or flowchart illustration, and combinations of blocks in the block diagrams and/or flowchart illustration, can be implemented by special purpose hardware-based systems that perform the specified functions or acts or carry out combinations of special purpose hardware and computer instructions.
The descriptions of the various embodiments of the present invention have been presented for purposes of illustration, but are not intended to be exhaustive or limited to the embodiments disclosed. Many modifications and variations will be apparent to those of ordinary skill in the art without departing from the scope and spirit of the described embodiments. The terminology used herein was chosen to best explain the principles of the embodiments, the practical application or technical improvement over technologies found in the marketplace, or to enable others of ordinary skill in the art to understand the embodiments disclosed herein.

Claims (17)

What is claimed is:
1. A system for identifying relationships among a group of indicators, comprising:
a memory having computer readable instructions; and
a processor for executing the computer readable instructions, the computer readable instructions including instructions for:
obtaining the group of indicators, including a number of indicators relating to an operation of a computing system;
creating pairs of indicators, wherein the pairs of indicators includes all possible combination of the group of indicators;
for each pair of indicators:
calculating a linear correlation score;
calculating a fitting function score;
determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score; and
storing the final correlation score in a relationship database;
creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators, wherein the graphical display is a correlation map that includes a node for each indicator and a connector between each related pair of indicators, wherein a pair of indicators are determined to be related if there final correlation score is non-zero.
2. The system of claim 1, wherein the group of indicators include a discretized time indicator.
3. The system of claim 1, wherein the computer readable instructions including instructions for standardizing each of the group of indicators prior to creating the pairs of indicators.
4. The system of claim 1, wherein the final correlation score is determined to be zero based on determining that an absolute value of the linear correlation score is less than a threshold number and that one minus the fitting function score is less than the threshold number.
5. The system of claim 4, wherein the final correlation score is determined to be a greater of the absolute value of the linear correlation score and one minus the fitting function score.
6. The system of claim 1, wherein the graphical display is a ranking map for a selected indicator from the group of indicators that includes nodes for each of the group of indicators that have final correlation scores with the selected indicator that exceeds a threshold value.
7. The system of claim 6, wherein a thickness of a connector between the nodes of the ranking map is positively correlated with the final correlation score associated with the pair of indicators associated with the nodes that are connected by the connector.
8. A method for identifying relationships among a group of indicators, the method comprising:
obtaining the group of indicators, including a number of indicators relating to an operation of a computing system;
creating pairs of indicators, wherein the pairs of indicators includes all possible combination of the group of indicators;
for each pair of indicators:
calculating a linear correlation score;
calculating a fitting function score;
determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score; and
storing the final correlation score in a relationship database;
creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators, wherein the graphical display is a correlation map that includes a node for each indicator and a connector between each related pair of indicators, wherein a pair of indicators are determined to be related if there final correlation score is non-zero.
9. The method of claim 8, wherein the group of indicators include a discretized time indicator.
10. The method of claim 8, further comprising standardizing each of the group of indicators prior to creating the pairs of indicators.
11. The method of claim 8, wherein the final correlation score is determined to be zero based on determining that an absolute value of the linear correlation score is less than a threshold number and that one minus the fitting function score is less than the threshold number.
12. The method of claim 11, wherein the final correlation score is determined to be a greater of the absolute value of the linear correlation score and one minus the fitting function score.
13. The method of claim 8, wherein the graphical display is a ranking map for a selected indicator from the group of indicators that includes nodes for each of the group of indicators that have final correlation scores with the selected indicator that exceeds a threshold value.
14. The method of claim 13, wherein a thickness of a connector between the nodes of the ranking map is positively correlated with the final correlation score associated with the pair of indicators associated with the nodes that are connected by the connector.
15. A computer program product comprising a non-transitory computer readable storage medium having program instructions embodied therewith the program instructions executable by a computer processor to cause the computer processor to perform a method, comprising:
obtaining a group of indicators, including a number of indicators relating to an operation of a computing system;
creating pairs of indicators, wherein the pairs of indicators includes all possible combination of the group of indicators;
for each pair of indicators:
calculating a linear correlation score;
calculating a fitting function score;
determining a final correlation score based at least in part on one of the linear correlation score and the fitting function score; and
storing the final correlation score in a relationship database;
creating a graphical display based on the relationship database, wherein the graphical display is configured to convey a strength relationships among the group of indicators, wherein the graphical display is a correlation map that includes a node for each indicator and a connector between each related pair of indicators, wherein a pair of indicators are determined to be related if there final correlation score is non-zero.
16. The computer program product of claim 15, wherein the final correlation score is determined to be zero based on determining that an absolute value of the linear correlation score is less than a threshold number and that one minus the fitting function score is less than the threshold number.
17. The computer program product of claim 16, wherein the final correlation score is determined to be a greater of the absolute value of the linear correlation score and one minus the fitting function score.
US16/019,656 2018-06-27 2018-06-27 Identifying relationships among a group of indicators Expired - Fee Related US10878001B2 (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
US16/019,656 US10878001B2 (en) 2018-06-27 2018-06-27 Identifying relationships among a group of indicators

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
US16/019,656 US10878001B2 (en) 2018-06-27 2018-06-27 Identifying relationships among a group of indicators

Publications (2)

Publication Number Publication Date
US20200004871A1 US20200004871A1 (en) 2020-01-02
US10878001B2 true US10878001B2 (en) 2020-12-29

Family

ID=69008171

Family Applications (1)

Application Number Title Priority Date Filing Date
US16/019,656 Expired - Fee Related US10878001B2 (en) 2018-06-27 2018-06-27 Identifying relationships among a group of indicators

Country Status (1)

Country Link
US (1) US10878001B2 (en)

Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2005015404A2 (en) 2003-08-06 2005-02-17 Moshe Halevy Method and apparatus for unified performance modeling with monitoring and analysis of complex systems
US7194421B2 (en) 2000-01-13 2007-03-20 Erinmedia, Llc Content attribute impact invalidation method
US20090063304A1 (en) * 2007-08-29 2009-03-05 Anthony Meggs System and method for searching, identifying, and ranking merchants based upon preselected criteria such as social values
US20090164913A1 (en) * 2007-12-21 2009-06-25 Jonathan Davar Supplementing user web-browsing
US20100106426A1 (en) * 2008-10-23 2010-04-29 Microsoft Corporation Regions of interest processing
US8095415B1 (en) * 2003-05-07 2012-01-10 Accenture Global Services Gmbh Human capital development framework
US8132122B2 (en) 2002-10-21 2012-03-06 Battelle Memorial Institute Multidimensional structured data visualization method and apparatus, text visualization method and apparatus, method and apparatus for visualizing and graphically navigating the world wide web, method and apparatus for visualizing hierarchies
US20120066618A1 (en) * 2010-04-14 2012-03-15 Linkedln Corporation Carousel of the new
US20140074843A1 (en) * 2012-09-12 2014-03-13 Zuess, Inc. Systems and methods for dynamic analysis, sorting and active display of semantic-driven reports of communication repositories
US20140079297A1 (en) * 2012-09-17 2014-03-20 Saied Tadayon Application of Z-Webs and Z-factors to Analytics, Search Engine, Learning, Recognition, Natural Language, and Other Utilities
US8849823B2 (en) 2011-10-20 2014-09-30 International Business Machines Corporation Interactive visualization of temporal event data and correlated outcomes
US20150142811A1 (en) * 2013-10-21 2015-05-21 Agile Legal Technology Content Categorization System
US20170013486A1 (en) * 2015-07-06 2017-01-12 Jds Uniphase Corporation Channel emulation for testing network resources
US9697470B2 (en) 2014-04-16 2017-07-04 Applied Materials, Inc. Apparatus and method for integrating manual and automated techniques for automated correlation in data mining
US9704143B2 (en) 2014-05-16 2017-07-11 Goldman Sachs & Co. LLC Cryptographic currency for securities settlement
US9916605B2 (en) 2015-06-27 2018-03-13 International Business Machines Corporation Collaboration group recommendations derived from request-action correlations
US20180101869A1 (en) * 2016-10-10 2018-04-12 Cellock Ltd Method and information system for enhanced traveler experience during travel
US20180197095A1 (en) * 2014-06-23 2018-07-12 Nicole Sponaugle Method for identifying countries vulnerable to unrest
US20190068659A1 (en) * 2007-12-21 2019-02-28 Jonathan Davar Supplementing user web-browsing

Patent Citations (19)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7194421B2 (en) 2000-01-13 2007-03-20 Erinmedia, Llc Content attribute impact invalidation method
US8132122B2 (en) 2002-10-21 2012-03-06 Battelle Memorial Institute Multidimensional structured data visualization method and apparatus, text visualization method and apparatus, method and apparatus for visualizing and graphically navigating the world wide web, method and apparatus for visualizing hierarchies
US8095415B1 (en) * 2003-05-07 2012-01-10 Accenture Global Services Gmbh Human capital development framework
WO2005015404A2 (en) 2003-08-06 2005-02-17 Moshe Halevy Method and apparatus for unified performance modeling with monitoring and analysis of complex systems
US20090063304A1 (en) * 2007-08-29 2009-03-05 Anthony Meggs System and method for searching, identifying, and ranking merchants based upon preselected criteria such as social values
US20090164913A1 (en) * 2007-12-21 2009-06-25 Jonathan Davar Supplementing user web-browsing
US20190068659A1 (en) * 2007-12-21 2019-02-28 Jonathan Davar Supplementing user web-browsing
US20100106426A1 (en) * 2008-10-23 2010-04-29 Microsoft Corporation Regions of interest processing
US20120066618A1 (en) * 2010-04-14 2012-03-15 Linkedln Corporation Carousel of the new
US8849823B2 (en) 2011-10-20 2014-09-30 International Business Machines Corporation Interactive visualization of temporal event data and correlated outcomes
US20140074843A1 (en) * 2012-09-12 2014-03-13 Zuess, Inc. Systems and methods for dynamic analysis, sorting and active display of semantic-driven reports of communication repositories
US20140079297A1 (en) * 2012-09-17 2014-03-20 Saied Tadayon Application of Z-Webs and Z-factors to Analytics, Search Engine, Learning, Recognition, Natural Language, and Other Utilities
US20150142811A1 (en) * 2013-10-21 2015-05-21 Agile Legal Technology Content Categorization System
US9697470B2 (en) 2014-04-16 2017-07-04 Applied Materials, Inc. Apparatus and method for integrating manual and automated techniques for automated correlation in data mining
US9704143B2 (en) 2014-05-16 2017-07-11 Goldman Sachs & Co. LLC Cryptographic currency for securities settlement
US20180197095A1 (en) * 2014-06-23 2018-07-12 Nicole Sponaugle Method for identifying countries vulnerable to unrest
US9916605B2 (en) 2015-06-27 2018-03-13 International Business Machines Corporation Collaboration group recommendations derived from request-action correlations
US20170013486A1 (en) * 2015-07-06 2017-01-12 Jds Uniphase Corporation Channel emulation for testing network resources
US20180101869A1 (en) * 2016-10-10 2018-04-12 Cellock Ltd Method and information system for enhanced traveler experience during travel

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
Duan, Lian et al., "Selecting the Right Correlation Measure for Binary Data" ACM Transactions on Knowledge Discovery from Data; vol. 9, No. 2, Article 13; Publication date: Sep. 2014; pp. 13:1-13:28.
Xiong, Hui et al., "Exploiting a Support-based Upper Bound of Pearson's Correlation Coefficient for Efficiently Identifying Strongly Correlated Pairs", Research Track Paper; KDD'04; Aug. 22-25, 2004; Seattle, Washington, USA, Copyright 2004 ACM; pp. 334-343.

Also Published As

Publication number Publication date
US20200004871A1 (en) 2020-01-02

Similar Documents

Publication Publication Date Title
US20200044912A1 (en) Computer system alert situation detection based on trend analysis
US10380272B2 (en) Update computer-implemented method, computer program product and computer system for generating or updating topology model of pressure pipe network
Baggio et al. Complex and chaotic tourism systems: towards a quantitative approach
US11074514B2 (en) Confidence intervals for anomalies in computer log data
US9644976B2 (en) Building missing movement path of an object
US9772896B2 (en) Identifying intervals of unusual activity in information technology systems
CN108229364B (en) Building contour generation method and device, computer equipment and storage medium
US11283597B2 (en) Blockchain technology
US20170111378A1 (en) User configurable message anomaly scoring to identify unusual activity in information technology systems
US11468349B2 (en) POI valuation method, apparatus, device and computer storage medium
US10339471B2 (en) Ensemble based labeling
CN114115348B (en) Switching method and device of inspection device of unmanned aerial vehicle, electronic equipment and storage medium
CN113253319A (en) Road network extraction and track deviation correction method and system based on vehicle GPS
US10257055B2 (en) Search for a ticket relevant to a current ticket
US11443837B2 (en) Generation of test data for a data platform
US10878001B2 (en) Identifying relationships among a group of indicators
US10796036B2 (en) Prediction of inhalable particles concentration
US20180260722A1 (en) Probability density ratio estimation
US20240117575A1 (en) Infrastructure diagnostic device, infrastructure diagnostic method, and recording medium
US20170053291A1 (en) Optimal time scale and data volume for real-time fraud analytics
US11620605B2 (en) Summarizing business process models
US20180108084A1 (en) Automated cognitive psychometric scoring
Gokgoz Anomaly detection using gans in opensky network
CN113847923A (en) Calculation method and device of estimated arrival time, electronic equipment and readable storage medium
CN114463452A (en) Region portrait generation method, device, electronic equipment and computer readable medium

Legal Events

Date Code Title Description
AS Assignment

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW Y

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHAKRA, AL;PAN, TIAN MING;TIAN, PENG FEI;AND OTHERS;SIGNING DATES FROM 20180626 TO 20180627;REEL/FRAME:046211/0087

Owner name: INTERNATIONAL BUSINESS MACHINES CORPORATION, NEW YORK

Free format text: ASSIGNMENT OF ASSIGNORS INTEREST;ASSIGNORS:CHAKRA, AL;PAN, TIAN MING;TIAN, PENG FEI;AND OTHERS;SIGNING DATES FROM 20180626 TO 20180627;REEL/FRAME:046211/0087

FEPP Fee payment procedure

Free format text: ENTITY STATUS SET TO UNDISCOUNTED (ORIGINAL EVENT CODE: BIG.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STPP Information on status: patent application and granting procedure in general

Free format text: NON FINAL ACTION MAILED

STPP Information on status: patent application and granting procedure in general

Free format text: NOTICE OF ALLOWANCE MAILED -- APPLICATION RECEIVED IN OFFICE OF PUBLICATIONS

STCF Information on status: patent grant

Free format text: PATENTED CASE

FEPP Fee payment procedure

Free format text: MAINTENANCE FEE REMINDER MAILED (ORIGINAL EVENT CODE: REM.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

LAPS Lapse for failure to pay maintenance fees

Free format text: PATENT EXPIRED FOR FAILURE TO PAY MAINTENANCE FEES (ORIGINAL EVENT CODE: EXP.); ENTITY STATUS OF PATENT OWNER: LARGE ENTITY

STCH Information on status: patent discontinuation

Free format text: PATENT EXPIRED DUE TO NONPAYMENT OF MAINTENANCE FEES UNDER 37 CFR 1.362

FP Lapsed due to failure to pay maintenance fee

Effective date: 20241229

OSZAR »