so thay started to use GPT models, so do i
Description of
Description of the recommendation system (ver. 0.1; 09.10.2020)
General purpose
The recommender system (system) is designed to form recommendations for the events presented in it for making decisions regarding possible actions based on the data model for historical events of a similar type.
Description (user case)
An example of work on the example of sports betting:
The user turns to the chat bot with a request for current possible events, the bot gives out a list of matches for which there are recommendations.
User selects a specific match
The bot displays a list of possible recommendations.
A recommendation is a link to an object in a specific bookmaker system where you can make a specific bet with a recommendation on what value to bet and% of the probability that this recommendation is correct.
In addition to each recommendation, you can issue a link when you click on which will open a set of analytics on the basis of which this recommendation is given
The elements
The work will require the implementation of several independent elements (subsystems)
For greater security, at the first stage, it is best to deploy all subsystems on one host — preferably a local one. In industrial implementation, it is possible to install subsystems on a cloud infrastructure
Scheme:
Sports Events Database
A normalized base suitable as a dataset for training a neural network. The base must contain the maximum number of event attributes.
Ideally, this should be a minute-by-minute recording of each event with all the attributes a la log broadcast. (id, date, time, event, event type, attendees, attendee attributes, result, result attributes, links to related events, etc.)
Separately for each event, there should be a general description of the setting factors (id, date, time, place, participants, lineups, uniform color, weather, related events, etc.)
A robot for filling this database (parser of historical events)
There are 2 ways to create such a base.
the first is the acquisition of access or the base itself for use
the second is the independent formation of such a base by collecting information in open sources.
consider the second option:
it is necessary to implement a robot parser to collect information about ongoing events both in real time and for a historical period (mainly for a historical period).
to work, it is necessary to identify possible sources for parsing, probably these can be sites leading text broadcasting events, sports almanacs, sites of sports clubs and sports federations. It is probably best to use foreign sources as they are less censored and have greater historical depth. But it is not exactly.
specific settings of the parser, the number and composition of attributes can be determined based on the analysis of probable sources.
the parser’s task is to collect and aggregate data for further use by the neural network for forecasting
the recommended format is an event record (id, date, time, place, participants, compositions, uniform color, weather, related events, etc.) and a minute / second event log (id, date, time, event, event type, participants, attendee attributes, result, result attributes, links to related events, etc.)
the minimum amount of data for training a neural network and forming a model can be determined at the stage of creating this neural network, but it is assumed that the larger the amount of accumulated data, the more accurate the forecast will be.
Parser of current sporting events (real time parser)
To generate current recommendations, you need to implement the current events parser.
The task of this parser is to collect and structure information about current events in real time. Maintain a feed of events and supplement the existing database with related events to transfer information to the neural network and to the model to improve the quality of forecasts.
It is also necessary to identify possible sources for parsing, these can be sites covering sports events, aggregators, sites of teams and federations.
The logic of work is similar to the parser for filling the historical database.
A separate task of the parser is to form a list of current and upcoming events, generate information for presentation to the user.
Parser of current possible bets and odds
this parser is designed to integrate with bookmaker systems and increase user engagement.
The task of this parser is, based on data on current events, to form a list of possible bets, odds and links to them for placement in the user interface to simplify betting
This parser can be replaced with the betting system API in case of a successful integration agreement
Neural network for analyzing current events and forming a forecast for specific rates
The task of the neural network is to generate a forecast for the currently selected event based on data on similar events from the database.
principle of operation:
it is necessary to train the network to select the most probable events, taking into account the largest possible number of available attributes in the database.
events can be grouped by similarity, or opposite
events are influenced by related events, participants in the event and time, both absolute and relative time (for example, 31 minutes of a match)
related events can be both previous and subsequent, if there is information about them in the database
participants can be both specific characters (players, coaches, etc.) and teams, groups, divisions, leagues, etc.
thus, the network must be able to analyze the event at multiple levels, taking into account the associated events present.
The result of each calculation should be a specific recommendation in relation to a specific possible bet on the bookmaker’s website for a specific event.
The recommendation may be limited in time or its likelihood of execution may decrease over time.
the result of calculations can probably be represented in the form of a graph — a set of all related events and participants with weights of influence on the final recommendation
each recommendation (calculation result) must be stored in a separate database for the possibility of reinforcement learning
a more detailed description of the learning process and the operation of a neural network requires the involvement of a specialist.
transaction machine (back-end)
The server side of the application, which provides management of the interaction of other modules (subsystems), schedule management, user management, permission management, as well as the formation of a view for interaction with the user.
The system should log all operations into a separate database or be able to connect a monitoring system (Splunk, Elastic, etc.)
The system should be able to create a schedule and connect sources for parsing by a historical parser in the administrator interface. It should be possible to launch several streams of the historical parser, either from one source or from several. You will probably need a module for depersonalizing access for parsers, providing credentials, changing the IP address and other tricks to improve the work of the historical parser.
The system must have one main thread of the real-time parser, and the ability to run several in parallel. The main real-time parser should automatically generate a schedule for further launches of the necessary modules to support the processing of detected events. You need to be able to adjust the schedule in the admin interface.
Based on the data from the real-time parser, tasks should be generated for the parser of bookmaker sites to receive bets on these events and a neural network for calculating recommendations for these events.
The calculations can be refined as the event approaches, and will be refined during the event in each new entry in the log for this event.
for good reason, for each upcoming event, it should launch a separate instance — an event handler. Probably, in order not to overload the system, events should be selected by the administrator in his interface.
that is, there are 10 matches in the week, at first, the administrator selects 3 key ones in the interface, this launches 3 instances of the neural network to analyze each event separately. The results of the analysis are added to separate databases, hypotheses are formed — recommendations in relation to the offers of bookmakers.
At the moment the first event starts, a separate instance of the process is launched to refine the hypothesis in real time, and the results of forecasts for upcoming events are used as related.
If 2 events occur in parallel, then 2 or more process instances must be launched simultaneously.
the system should form a set of data suitable for the formation of the user interface.
the main elements are user identification, requesting current possible recommendations, viewing a specific selected recommendation, viewing a decision graph for a specific selected recommendation.
You need to be able to differentiate views depending on the role or user level, you need to be able to manage the user level and role in the administrator interface.
You will need to connect the payment system to be able to pay the user for the subscription, both directly through the user interface, and possibly through the Apple, Google app stores …
user interface (frond-end)
The interface should provide the user with the following capabilities:
New User Registration
Subscription payment
the possibility of using a discount coupon
renewal option
Authentication (probably over the phone?)
Access recovery
view current possible recommendations and rates for them
an overview of upcoming events for which recommendations will be offered (if selected by the administrator)
viewing a specific recommendation for a specific event
viewing the decision graph for a specific recommendation
go to the bookmaker’s website to place bets
the ability to contact support in case of problems
additionally:
it will probably be possible to add social elements, such as voting for specific recommendations, discussion and likes of specific recommendations, group chats, chat with support.
The initial concept of the interface is implementation in the form of a telegram bot; in the future, implementation in the form of a separate application or website is possible.
admin management interface
The admin interface should provide the following capabilities:
Authentication
viewing the current state of the system (dashboard with a description of the status of the historical parser, real-time parser, the number of parallel processes, processed sources, possibly something else)
viewing and editing the current calendar, upcoming and past events, with the ability to fall into events
the ability to view events with a list of related recommendations, their graphs). Ability to select events for building recommendations and excluding events.
the ability to create your own events with a specific set of attributes (probably required for testing)
viewing a list of users, their role, level, status, etc. with the ability to fail in each
viewing and editing a specific user, his role, level, etc.
viewing statistics on system operation (number of connections, connection details, traffic consumption, traffic sources, protocols, hardware load, operation logs, etc.)
Sources of data:
http://sport-calendar.ru/
https://understat.com/
https://www.footballoutsiders.com/
http://www.advancedfootballanalytics.com/
https://777score.ng/
https://hintwise.com/?league=&date=2020-10-09&show=all
Football-data.co.uk Historical football results and betting odds data
Soccerway Current and past match results
RSSSF Rec.Sport.Soccer Statistics Foundation
Datasets:
https://www.kaggle.com/austro/beat-the-bookie-worldwide-football-dataset (downloaded, in the folder)
https://www.kaggle.com/sashchernuh/european-football (downloaded in the folder)
https://data.world/datasets/betting
https://datahub.io/collections/football
Examples:
https://x-betting.org/programmy-dlya-stavok-na-sport — ready-made solutions
http://sport-analiz.com/ — turnkey solution
https://sportsflare.io/ — turnkey solution
https://apps.apple.com/ru/app/%D0%BF%D1%80%D0%BE%D0%B3%D0%BD%D0%BE%D0%B7%D1%8B-%D0% BD% D0% B0-% D1% 81% D0% BF% D0% BE% D1% 80% D1% 82-% D0% B8-% D1% 84% D1% 83% D1% 82% D0% B1% D0 % BE% D0% BB / id1473274550 — iOS app
https://excelvba.ru/programmes/Parser/samples/category/Sport — parsers, developer
https://sportsoft.ru/publications/analiz-dannyh-v-sporte-zapadnyj-opyt-ispolzovaniya-26 — developers
Attempts:
https://kushvsporte.ru/all
https://stavkiprognozy.ru/prognozy/
https://stavkiprognozy.ru/
system (ver. 0.1; 09.10.2020)
General purpose
The recommender system (system) is designed to form recommendations for the events presented in it for making decisions regarding possible actions based on the data model for historical events of a similar type.
Description (user case)
An example of work on the example of sports betting:
The user turns to the chat bot with a request for current possible events, the bot gives out a list of matches for which there are recommendations.
User selects a specific match
The bot displays a list of possible recommendations.
A recommendation is a link to an object in a specific bookmaker system where you can make a specific bet with a recommendation on what value to bet and% of the probability that this recommendation is correct.
In addition to each recommendation, you can issue a link when you click on which will open a set of analytics on the basis of which this recommendation is given
The elements
The work will require the implementation of several independent elements (subsystems)
For greater security, at the first stage, it is best to deploy all subsystems on one host — preferably a local one. In industrial implementation, it is possible to install subsystems on a cloud infrastructure
Scheme:
Sports Events Database
A normalized base suitable as a dataset for training a neural network. The base must contain the maximum number of event attributes.
Ideally, this should be a minute-by-minute recording of each event with all the attributes a la log broadcast. (id, date, time, event, event type, attendees, attendee attributes, result, result attributes, links to related events, etc.)
Separately for each event, there should be a general description of the setting factors (id, date, time, place, participants, lineups, uniform color, weather, related events, etc.)
A robot for filling this database (parser of historical events)
There are 2 ways to create such a base.
the first is the acquisition of access or the base itself for use
the second is the independent formation of such a base by collecting information in open sources.
consider the second option:
it is necessary to implement a robot parser to collect information about ongoing events both in real time and for a historical period (mainly for a historical period).
to work, it is necessary to identify possible sources for parsing, probably these can be sites leading text broadcasting events, sports almanacs, sites of sports clubs and sports federations. It is probably best to use foreign sources as they are less censored and have greater historical depth. But it is not exactly.
specific settings of the parser, the number and composition of attributes can be determined based on the analysis of probable sources.
the parser’s task is to collect and aggregate data for further use by the neural network for forecasting
the recommended format is an event record (id, date, time, place, participants, compositions, uniform color, weather, related events, etc.) and a minute / second event log (id, date, time, event, event type, participants, attendee attributes, result, result attributes, links to related events, etc.)
the minimum amount of data for training a neural network and forming a model can be determined at the stage of creating this neural network, but it is assumed that the larger the amount of accumulated data, the more accurate the forecast will be.
Parser of current sporting events (real time parser)
To generate current recommendations, you need to implement the current events parser.
The task of this parser is to collect and structure information about current events in real time. Maintain a feed of events and supplement the existing database with related events to transfer information to the neural network and to the model to improve the quality of forecasts.
It is also necessary to identify possible sources for parsing, these can be sites covering sports events, aggregators, sites of teams and federations.
The logic of work is similar to the parser for filling the historical database.
A separate task of the parser is to form a list of current and upcoming events, generate information for presentation to the user.
Parser of current possible bets and odds
this parser is designed to integrate with bookmaker systems and increase user engagement.
The task of this parser is, based on data on current events, to form a list of possible bets, odds and links to them for placement in the user interface to simplify betting
This parser can be replaced with the betting system API in case of a successful integration agreement
Neural network for analyzing current events and forming a forecast for specific rates
The task of the neural network is to generate a forecast for the currently selected event based on data on similar events from the database.
principle of operation:
it is necessary to train the network to select the most probable events, taking into account the largest possible number of available attributes in the database.
events can be grouped by similarity, or opposite
events are influenced by related events, participants in the event and time, both absolute and relative time (for example, 31 minutes of a match)
related events can be both previous and subsequent, if there is information about them in the database
participants can be both specific characters (players, coaches, etc.) and teams, groups, divisions, leagues, etc.
thus, the network must be able to analyze the event at multiple levels, taking into account the associated events present.
The result of each calculation should be a specific recommendation in relation to a specific possible bet on the bookmaker’s website for a specific event.
The recommendation may be limited in time or its likelihood of execution may decrease over time.
the result of calculations can probably be represented in the form of a graph — a set of all related events and participants with weights of influence on the final recommendation
each recommendation (calculation result) must be stored in a separate database for the possibility of reinforcement learning
a more detailed description of the learning process and the operation of a neural network requires the involvement of a specialist.
transaction machine (back-end)
The server side of the application, which provides management of the interaction of other modules (subsystems), schedule management, user management, permission management, as well as the formation of a view for interaction with the user.
The system should log all operations into a separate database or be able to connect a monitoring system (Splunk, Elastic, etc.)
The system should be able to create a schedule and connect sources for parsing by a historical parser in the administrator interface. It should be possible to launch several streams of the historical parser, either from one source or from several. You will probably need a module for depersonalizing access for parsers, providing credentials, changing the IP address and other tricks to improve the work of the historical parser.
The system must have one main thread of the real-time parser, and the ability to run several in parallel. The main real-time parser should automatically generate a schedule for further launches of the necessary modules to support the processing of detected events. You need to be able to adjust the schedule in the admin interface.
Based on the data from the real-time parser, tasks should be generated for the parser of bookmaker sites to receive bets on these events and a neural network for calculating recommendations for these events.
The calculations can be refined as the event approaches, and will be refined during the event in each new entry in the log for this event.
for good reason, for each upcoming event, it should launch a separate instance — an event handler. Probably, in order not to overload the system, events should be selected by the administrator in his interface.
that is, there are 10 matches in the week, at first, the administrator selects 3 key ones in the interface, this launches 3 instances of the neural network to analyze each event separately. The results of the analysis are added to separate databases, hypotheses are formed — recommendations in relation to the offers of bookmakers.
At the moment the first event starts, a separate instance of the process is launched to refine the hypothesis in real time, and the results of forecasts for upcoming events are used as related.
If 2 events occur in parallel, then 2 or more process instances must be launched simultaneously.
the system should form a set of data suitable for the formation of the user interface.
the main elements are user identification, requesting current possible recommendations, viewing a specific selected recommendation, viewing a decision graph for a specific selected recommendation.
You need to be able to differentiate views depending on the role or user level, you need to be able to manage the user level and role in the administrator interface.
You will need to connect the payment system to be able to pay the user for the subscription, both directly through the user interface, and possibly through the Apple, Google app stores …
user interface (frond-end)
The interface should provide the user with the following capabilities:
New User Registration
Subscription payment
the possibility of using a discount coupon
renewal option
Authentication (probably over the phone?)
Access recovery
view current possible recommendations and rates for them
an overview of upcoming events for which recommendations will be offered (if selected by the administrator)
viewing a specific recommendation for a specific event
viewing the decision graph for a specific recommendation
go to the bookmaker’s website to place bets
the ability to contact support in case of problems
additionally:
it will probably be possible to add social elements, such as voting for specific recommendations, discussion and likes of specific recommendations, group chats, chat with support.
The initial concept of the interface is implementation in the form of a telegram bot; in the future, implementation in the form of a separate application or website is possible.
admin management interface
The admin interface should provide the following capabilities:
Authentication
viewing the current state of the system (dashboard with a description of the status of the historical parser, real-time parser, the number of parallel processes, processed sources, possibly something else)
viewing and editing the current calendar, upcoming and past events, with the ability to fall into events
the ability to view events with a list of related recommendations, their graphs). Ability to select events for building recommendations and excluding events.
the ability to create your own events with a specific set of attributes (probably required for testing)
viewing a list of users, their role, level, status, etc. with the ability to fail in each
viewing and editing a specific user, his role, level, etc.
viewing statistics on system operation (number of connections, connection details, traffic consumption, traffic sources, protocols, hardware load, operation logs, etc.)
Sources of data:
http://sport-calendar.ru/
https://understat.com/
https://www.footballoutsiders.com/
http://www.advancedfootballanalytics.com/
https://777score.ng/
https://hintwise.com/?league=&date=2020-10-09&show=all
Football-data.co.uk Historical football results and betting odds data
Soccerway Current and past match results
RSSSF Rec.Sport.Soccer Statistics Foundation
Datasets:
https://www.kaggle.com/austro/beat-the-bookie-worldwide-football-dataset (downloaded, in the folder)
https://www.kaggle.com/sashchernuh/european-football (downloaded in the folder)
https://data.world/datasets/betting
https://datahub.io/collections/football
Комментарии закрыты.