Expertise Discovery And Credentialing For Evidence-driven Decision-making Tool

[0001] Electronic discovery, commonly referred to as e-discovery or eDiscovery, refers to any process in which electronic data is sought, located, secured and searched with the intent of using it as evidence in a business process, an audit, a regulatory investigation, an investigation or the like. E-discovery can be carried out offline on a particular computer or it can be accomplished in a network environment.

[0002] The nature of digital data makes it extremely well-suited for investigation. In particular, digital data can be electronically searched with ease, while paper documents must be scrutinized manually. Furthermore, it is difficult or impossible to completely destroy digital data, particularly if the data is stored in a network environment. This is because the data appears on multiple hard drives, and because digital files, even if deleted, generally can be undeleted. In fact, the only reliable means of destroying digital data is to physically destroy any and all hard drives where it is stored. More importantly, if the data is stored in a distributed framework like that in a blockchain system and owned/verified through smart contracts, it is not easy to destroy all copies of the data (which serves as an evidence that cannot be tampered).

[0003] In the process of electronic discovery, data of all types can serve as evidence. This can include text, image, calendar event data, databases, spreadsheets, audio files, multimedia files, web sites and computer programs.

[0004] Advancements in the field of E-discovery has led to a growth across different business applications. The author of this disclosure utilized a variety of technological frameworks to use evidences generated from e discovery processes and extract credentialed and verified profiles of experts across different domains that can be trusted because of the supporting facts and evidences.

[0005] As the complexity of scientific endeavours increases almost exponentially, finding the best reviewers for evaluating innovations such as grant submissions, evaluating complex innovations for portfolio selection decisions such as funding, and tracking real-world utilization of inventions is getting harder and harder.

[0006] When social networks are used in a professional environment, one issue that presents itself is improving finding experts to enhance business and scientific operations. As organizations become increasingly global and telecommunications based, the geographical location of individual employees becomes less consequential in the daily conduct of business operations. By tapping into a global view of how people are connected in an organization, tools based on social network analysis can enhance long-range information-flow and knowledge sharing within the enterprise, beyond what can be offered by local access to other employees and traditional information sources such as intranet web pages.

[0007] A number of techniques have been proposed for finding experts in social and professional networks. Generally, techniques for finding experts in social networks have involved matching a question with expert profiles, sometimes taking into account expert availability. But all these portals and networks have limitations and generally do not provide necessary detailed understanding about experts that can be trusted. More importantly, this information lacks evidence.

[0008] One example is to be found in J. Zhang et al., “Expertise Networks in Online Communities: Structure and Algorithms,” International World Wide Web Conference Committee (IW3C2), (May 8-12, 207, Banff, Alberta, Canada; ACM 978-1-59593-65-7/07/0005 (“Zhang et al.”). Zhang et al. concluded that methods like page-rank that work for ranking web-pages, do not work very well for ranking experts due to structural differences in the web graph and the online user-expert network respectively. They simulated the dynamics of question-answering and found that expertise-networks are quite different structurally because users and experts do not behave in an identical manner.

[0009] Unlike many decisions made today based on qualitative or intuitive approaches due to limited resources and evidences about experts, there is a need of a system that provides a set of tools and digital technologies that recommend actionable decisions to grant reviewers and innovation evaluators. There is a need of a unified, trusted system that enables expertise discovery and credentialing for evidence-driven decision-making.

[0010] FIG. 1 illustrates an ecosystem 100 for facilitating finding of select reviewers for purposes of innovation evaluation in accordance with an example. The FIG. 1 includes an ‘Evidence-Based Decision Aid’ 112 in accordance with an example.

[0011] FIG. 2 illustrates exemplary details about an expert profile 200 extracted by the EDA 112 using a crowdsourced credentialing and discovery process in accordance with an example from an information source such as 102a. Other information sources may generate more details and together all information can convert into a unified profile of the expert that is more trusted as compared to a single source-based non credentialed profile view.

[0012] The ecosystem 100 illustrated in FIG. 1 facilitates finding of reviewers for innovation evaluation purposes in accordance with an example for predefined topics of interest. As shown in the FIGURE 1, the ecosystem 100 includes the Evidence-Based Decision Aid (EDA) 112 which comprises a set of tools and digital technologies 106 that recommend actionable decisions to grant reviewers and innovation evaluators in identifying the right kinds of expertise from a pool of resources located globally through crowdsourced credentialing and peer reviews kind of process executed through computer-enabled systems and methods.

[0013] The EDA 112 further includes a processing circuit 108 which takes inputs from the set of tools and digital technologies 104 and processes these inputs to generate computer-executable actions for facilitating identification of the select reviewers to make certain business decisions through computer-controlled operations. Each input generated by the processing circuit 108 is substantiated with evidences as opposed to pure subjective assessments about the identified experts.

[0014] FIG. 1 also shows various components of the EDA 112 without limitations, in an example. Referring to the FIG. 1 the EDA 112 and a method of facilitating selection of the reviewers for specified decisions based on evidences are discussed herein.

[0015] In an example, the EDA 112 will help innovation portfolio managers at small and large institutions and government agencies route ideation reviews to the right personnel, allow personnel to review ideas and grants faster, and encourage reviewers to choose innovations more equitably (what we refer to as evidence-driven decision-making) based on outcomes generated by the processing circuit 108 of the EDA 112.

[0016] In an example, the EDA 112 will allow users searching for experts in specific scientific and technological fields to find the best credentialed, experienced, and skilled individuals for those subject areas that are well beyond what’s available in existing proprietary databases, public sources such as PubMed, or professional social networks such as LinkedInTM. Existing sources rarely identify granular scientific or engineering expertise nor do they relate to perceived or actual conflicts of interest and instead provide general information pertinent to job screening rather than a grant or innovation review which requires substantially more detailed information about their expertise and skills for specific innovation subject areas.

[0017] The EDA 112 may also enable experts to review each other (which the system or EDA 112 may validate through active phone calls and digital polling without limitations), which surfaces conflicts and provides further transparency of granularity of scientific and engineering expertise (which the system 112 will validate through physical and virtual means), so that users seeking experts can see what their peers’ objective opinions about their skills, conflicts, and expertise in any given subject area (similar to how peers review each other’s scientific papers).

[0018] By transforming their overall review process utilizing crowdsourced reviews through expert-driven, trusted, and digitally-controlled global networks, decision-making burdens of applicants, reviewers, and other staff members associated with those agencies should be greatly reduced. More importantly, though the quality of the decisions should be improved.

[0019] In an example, as shown in FIG. 1, the EDA 112 may be connected digitally over a network 110 to a plurality of information sources including such as a first information source 102a containing a first digitally executable asset 104a associated with an expert, a second information source 102b containing a second digitally executable asset 104b associated with the expert, a third information source 102c containing a third digitally executable asset 104c associated with the expert, a fourth information source 102d containing a fourth digitally executable asset 104d associated with the expert. The plurality of information sources together referred to as 102 provides specific information about the expert stored in their respective digitally executable assets together referred to as 104. The digitally executable assets 104 may be stored at discrete databases and defined in the form of profile components of the expert which together may serve as a source of information for further validation about the expert. The FIG. 1 only shows tracing of information about one expert, but a large number of experts and reviewers may be traced and discovered through a similar process over the network 110 without limitations.

[0020] The EDA 112 may include an operating system not shown in FIGURE that includes procedures for handling various basic system services and for performing hardware dependent tasks; a communication module (not shown) that is used for connecting the EDA 112 to other computers (e.g., the information sources 102) via one or more communication interfaces (wired or wireless) over the network 110, which in turn connect the EDA 112 to one or more communications networks such as the Internet, other wide area networks, local area networks, blockchain network, and so on.

[0021] The EDA 112 may include a crawling module 114 for searching and crawling Web sites and other types of information sources that contain relevant digitally executable assets 104 about the experts. The crawling module 114 may include a URL queue that stores URLs to be crawled by crawler periodically, a link parser that tells crawler how to extract new links to follow from a fetched page; a crawler that fetches web pages, and a buffer that temporarily stores web pages and other links associated with the information sources fetched by the crawler.

[0022] In addition to extracting information relevant for the expert, the EDA 112 may also fetch information about reviews, comments, cross-references etc. provided by peers and others about the expert which together may be termed herein as ‘reviews’ or ‘peer reviews’. The reviews can be extracted by a review extraction module 116 that may be contained within a crowdsourced credentialing engine 118.

[0023] In an example, the EDA 112 may include a sentiment application 120 that may utilize information contained in the information sources 102 about the expert, apply sentiment lexicon and business logic and heuristics, with the help of AI/Machine Learning tools to determine sentiment scores about the extracted information. The sentiment application 120 may further generate polarity indices or polarity identifiers indicative of positive and negative polarities or expressions about specific expertise and skillsets of the experts by peers based on the crowdsourced credentialing / reviews obtained by the EDA 112.

[0024] The EDA 112 may generate a review map indicative of a review index about the plurality of experts tied to a specific expertise topic that can assist in visualizing credentialed expertise, reviews and capabilities of different experts over a graphical interface or dashboard easily by people or machine-dependent tools, who are involved in taking decisions about hiring people for grants and innovation reviews. The EDA 112 can also generate hot spots over the map that may allow hiring agencies to easily identify right targets or extremely bad possibilities. Specific kinds of alerts and warnings may be integrated into the system 112 to keep people updated about opportunities and threats.

[0025] The central purpose of the EDA 112 is to allow evidence to drive the decisions as opposed to opinions which may become a reason of failure of scientific endeavours. Credentialed expertise and aggregation of reviews from a variety of information sources provide necessary evidence that can be trusted. The more information sources the EDA 112 looks out to gather information about an expert, the more strengthened is the evidence to believe upon.

[0026] Each of the identified modules and applications/tools discussed herein corresponds to a set of instructions for performing a function as described herein. These modules (i.e., sets of instructions) need not be implemented as separate software programs, procedures or modules, and thus various subsets of these modules may be combined or otherwise re-arranged in various embodiments. In some embodiments, a memory may store a subset of the modules and data structures identified herein. Furthermore, the memory may store additional modules and data structures not described herein.

[0027] In some embodiments, collecting information about the expert comprises receiving reviews by peers in a predetermined digital format, for example as data streams from a plurality of reviewers/peers. The predetermined digital format of the reviews may comprise data organized in predefined data structures. In some embodiments, collecting information comprises selecting information from a repository of information collected by the web crawler or crawling module 114 of the EDA 112.

[0028] The crowdsourced credentialing engine 118 of the EDA 112 may be similar to the one proposed in US9870591B2 by Shahid N. Shah. The crowdsourced credentialing engine 118 allows a plurality of crowdsourced respondents to respond to segmented digital profiles associated with each of the plurality of experts as identified by the EDA 112 based on information collected from the plurality of information sources 102 and credential the experts to determine crowdsourced credentialed expertise, wherein the credentialing of each of the segmented digital profiles associated with the expert of the plurality of experts contribute to credentialing of their overall digital profiles.

[0029] If the profiles and reviews about an expert are not a good match for his/her expertise as required for evaluation of a particular innovation or grant, agencies involved in making decisions can quickly establish this fact and take appropriate action. In an example, the decision may be taken manually be a user or a group of users but other steps can be performed utilizing technologies and computer-implemented systems. In an example, even the final decision can be taken programmatically and an outcome can be reported to respective experts about the decisions made by the system 112.

[0030] At the end of the outcome, both parties can optionally rate their satisfaction with the interaction on a scale of 1 to 5 which then could be used back into the system for validation of the outcomes. The complete interaction may be recorded in a database 122 on a server.

[0031] In an example, the EDA 112 may include computing a similarity score between multiple discovered experts or reviewers. If one expert is discovered and evaluated as suitable for a job, based on the similarity scores, a ranking of more suitable experts as compared to others may be determined to automatically surface candidates that are likely going to be accepted for the job as opposed to screening non related or less related profiles.

[0032] The similarity score may be calculated based on a function of a profile of the expert and a set of variables that may be predefined and indicative of relevant parameters. For example, for one job association with surgical departments may be more relevant than other parameters and can be considered as a relevant variable to identify similarity score among the experts. In some cases, peer review questions may be defined and answers from experts to those questions may generate similarity scores. The score may be further based on a probabilistic score based on the variable identifying the expert and/or using recommendation engines.

[0033] In an example, the EDA 112 may be implemented in a blockchain-configured ecosystem architecture containing one or more additional components so as to allow maintaining integrity of submission of the reviews, profiles, and other evidences. This may facilitate association of an identity to the evidences thus submitted and also define and store a geographical and temporal identity such as when and in which location the participating experts are identified. The blockchain configured ecosystem architecture may provide a crowdsourced integrity network for storing the profiles of the experts instead of locally stored information by different participants that may be tampered.

[0034] The blockchain may eliminate risks associated with data being held centrally by storing data about the reviews and profiles across the network which may include the digitally executable assets 104 extracted by the EDA 112 from the plurality of information sources 102.

[0035] Although the embodiments of the present disclosure have been described in detail, it should be understood that various changes and substitutions can be made therein without departing from spirit and scope as defined by the disclosure herein. Variations described for the present disclosure can be realized in any combination desirable for each particular application. Thus particular limitations, and/or embodiment enhancements described herein, which may have particular advantages to a particular application need not be used for all applications. Also, not all limitations need be implemented in methods, systems and/or apparatus including one or more concepts of the present disclosure.

[0036] The present disclosure can be realized in hardware, software, or a combination of hardware and software. A typical combination of hardware and software could be a specific purpose computer system with a computer program that, when being loaded and run, controls the computer system such that it carries out the methods described herein. The present invention can also be embedded in a computer program product, which comprises all the features enabling the implementation of the methods described herein, and which—when loaded in a computer system is able to carry out these methods.

FIG. 1

FIG. 2