Proposal for an An AI Penny Auction Agent

2014-09-29 00:00:00 +0000

Proposal:​ ​Learning​ ​Agent​ ​for​ ​Penny-Auction​ ​Games

Main Research Questions and Core Research Objectives

Since​ ​the​ ​inception​ ​of​ ​the​ ​popularized​ ​“penny-auction”​ ​in​ ​the​ ​early​ ​2000s,​ ​there​ ​has​ ​been much​ ​study​ ​to​ ​show​ ​that​ ​the​ ​base​ ​form​ ​of​ ​bidding​ ​fee​ ​auctions,​ ​as​ ​they​ ​are​ ​alternatively​ ​called,​ ​is essentially​ ​gambling.​ ​The​ ​simplest​ ​form​ ​of​ ​these​ ​games​ ​involves​ ​buying​ ​the​ ​​ability​​ ​to​ ​bid,​ ​known henceforth​ ​as​ ​bid​ ​credits,​ ​and​ ​with​ ​each​ ​bid​ ​credit​ ​used​ ​the​ ​auction​ ​price​ ​of​ ​the​ ​good​ ​for​ ​sale increases​ ​by​ ​a​ ​nominal​ ​amount,​ ​which​ ​is​ ​often​ ​one​ ​cent.​ ​Therefore​ ​the​ ​winning​ ​bidder​ ​ends​ ​up paying​ ​for​ ​the​ ​number​ ​of​ ​bid​ ​credits​ ​purchased​ ​and​ ​the​ ​final​ ​sale​ ​price​ ​of​ ​the​ ​good,​ ​which​ ​is usually​ ​considerably​ ​lower​ ​than​ ​its​ ​market​ ​value.​ ​The​ ​auctioneer​ ​on​ ​the​ ​other​ ​hand,​ ​takes​ ​home these​ ​costs​ ​to​ ​the​ ​winning​ ​bidder​ ​in​ ​addition​ ​to​ ​all​ ​of​ ​the​ ​bid​ ​credits​ ​used​ ​by​ ​the​ ​other​ ​​n​​ ​bidders. Acknowledging​ ​these​ ​similarities​ ​with​ ​a​ ​pure​ ​betting​ ​game,​ ​there​ ​have​ ​been​ ​many​ ​fundamental changes​ ​in​ ​the​ ​business​ ​models​ ​of​ ​these​ ​penny-auction​ ​sites​ ​over​ ​the​ ​years​ ​that​ ​merit​ ​additional research​ ​into​ ​potential​ ​strategies​ ​of​ ​playing​ ​these​ ​games,​ ​the​ ​profitability​ ​of​ ​the​ ​auctioneer​ ​in relation​ ​to​ ​gamification​ ​elements,​ ​and​ ​the​ ​asymmetry​ ​of​ ​information​ ​present​ ​for​ ​players.

In​ ​this​ ​proposal,​ ​we’ll​ ​focus​ ​efforts​ ​on​ ​one​ ​of​ ​most​ ​popular​ ​auction​ ​sites,​ ​DealDash,​ ​for much​ ​of​ ​our​ ​research​ ​and​ ​analysis.​ ​Since​ ​the​ ​time​ ​of​ ​publication​ ​of​ ​much​ ​of​ ​the​ ​existing literature​ ​on​ ​penny-auctions,​ ​the​ ​key​ ​changes​ ​to​ ​their​ ​implementations​ ​may​ ​be​ ​justified​ ​by​ ​the theory​ ​behind​ ​these​ ​types​ ​of​ ​auctions.​ ​In​ ​particular,​ ​DealDash,​ ​which​ ​is​ ​the​ ​acquirer​ ​of​ ​the heavily​ ​studied​ ​“Swoopo”​ ​domain​ ​[1],​ ​has​ ​mechanisms​ ​that​ ​minimize​ ​the​ ​gambling​ ​elements​ ​of penny​ ​auctions​ ​by​ ​reducing​ ​risk​ ​for​ ​players.​ ​One​ ​such​ ​element​ ​is​ ​the​ ​Buy-It-Now​ ​feature,​ ​which if​ ​you​ ​lose​ ​an​ ​auction,​ ​lets​ ​you​ ​get​ ​all​ ​of​ ​your​ ​placed​ ​bid​ ​credits​ ​back​ ​so​ ​long​ ​as​ ​you​ ​buy​ ​an instance​ ​of​ ​the​ ​good​ ​that​ ​you​ ​were​ ​bidding​ ​on​ ​for​ ​its​ ​full​ ​market​ ​value.​ ​Another​ ​is​ ​the​ ​“no jumpers”​ ​auction,​ ​which​ ​doesn’t​ ​allow​ ​new​ ​bidders​ ​to​ ​enter​ ​an​ ​auction​ ​once​ ​the​ ​auction​ ​price​ ​has passed​ ​$5.00.​ ​The​ ​latter​ ​mechanism​ ​in​ ​particular​ ​goes​ ​a​ ​long​ ​way​ ​to​ ​increase​ ​symmetry​ ​in information​ ​with​ ​regard​ ​to​ ​how​ ​many​ ​players​ ​are​ ​in​ ​an​ ​auction,​ ​a​ ​key​ ​research​ ​area​ ​expanded upon​ ​by​ ​Mitzenmacher,​ ​Byers,​ ​and​ ​Zervas​ ​[1].

As​ ​a​ ​student​ ​of​ ​both​ ​computer​ ​science​ ​and​ ​economics,​ ​this​ ​topic​ ​offers​ ​a​ ​fascinating research​ ​opportunity​ ​for​ ​me,​ ​as​ ​I​ ​believe​ ​strategies,​ ​which​ ​​may​​ ​be​ ​conceptually​ ​unintuitive​ ​to humans,​ ​might​ ​exist​ ​given​ ​the​ ​implementation​ ​of​ ​these​ ​“new”​ ​business​ ​tactics.​ ​This​ ​combined with​ ​the​ ​fact​ ​that​ ​there​ ​is​ ​also​ ​far​ ​more​ ​data​ ​available​ ​on​ ​these​ ​sources​ ​than​ ​ever​ ​before,​ ​gives​ ​me a​ ​yearning​ ​for​ ​data-driven​ ​strategies​ ​to​ ​playing​ ​the​ ​penny-auction​ ​game.​ ​More​ ​specifically,​ ​I​ ​am interested​ ​in​ ​training​ ​a​ ​reinforcement​ ​learning​ ​or​ ​deep​ ​learning​ ​agent​ ​to​ ​play​ ​the​ ​game​ ​of​ ​penny auctions.

Scientific​ ​Contributions​ ​and​ ​Knowledge​ ​Gaps

Much​ ​of​ ​the​ ​existing​ ​research​ ​surrounding​ ​penny​ ​auctions​ ​focuses​ ​on​ ​finding​ ​equilibrium behavior​ ​for​ ​players​ ​[2,3,4]​ ​and​ ​it​ ​can​ ​go​ ​a​ ​long​ ​way​ ​to​ ​help​ ​train​ ​an​ ​agent​ ​to​ ​participate​ ​in penny-auctions​ ​before​ ​the​ ​agent​ ​even​ ​“sees​ ​the​ ​field”,​ ​or​ ​actually​ ​participates​ ​in​ ​a​ ​penny-auction itself.​ ​To​ ​this​ ​extent,​ ​the​ ​findings​ ​of​ ​other​ ​researchers​ ​can​ ​help​ ​initialize​ ​the​ ​premises​ ​or​ ​beliefs of​ ​an​ ​agent,​ ​rather​ ​than​ ​have​ ​it​ ​start​ ​with​ ​no​ ​knowledge​ ​on​ ​how​ ​to​ ​play​ ​the​ ​game.​ ​According​ ​to Mitzenmacher​ ​et​ ​al.,​ ​much​ ​of​ ​a​ ​player’s​ ​chance​ ​of​ ​winning,​ ​and​ ​inversely,​ ​the​ ​auctioneer’s profitability,​ ​is​ ​derived​ ​from​ ​the​ ​amount​ ​of​ ​information​ ​one​ ​has​ ​about​ ​the​ ​other​ ​bidders​ ​in​ ​the game.​ ​Knowing​ ​the​ ​number​ ​of​ ​other​ ​players,​ ​who​ ​they​ ​are,​ ​and​ ​their​ ​play​ ​histories​ ​is​ ​a​ ​crucial component​ ​for​ ​the​ ​strategy​ ​of​ ​playing​ ​the​ ​bidding​ ​game,​ ​and​ ​a​ ​machine​ ​can​ ​potentially​ ​interpret and​ ​calculate​ ​each​ ​of​ ​these​ ​tasks​ ​with​ ​much​ ​more​ ​speed​ ​and​ ​efficacy​ ​when​ ​compared​ ​to​ ​a​ ​human agent.​ ​My​ ​assumption​ ​here​ ​is​ ​also​ ​that​ ​human​ ​agents​ ​do​ ​not​ ​make​ ​their​ ​strategies​ ​based​ ​on​ ​which particular​ ​usernames​ ​are​ ​in​ ​the​ ​bidding​ ​pool​ ​with​ ​them.

It’s​ ​important​ ​to​ ​recognize​ ​where​ ​current​ ​research​ ​ends​ ​and​ ​what​ ​is​ ​needed​ ​however. Before​ ​starting​ ​we​ ​recognize​ ​that​ ​“fitting​ ​data​ ​is​ ​an​ ​improper​ ​approach​ ​for​ ​[model]​ ​validation,​ ​as many​ ​models​ ​with​ ​very​ ​different​ ​characteristics​ ​lead​ ​to​ ​power​ ​law​ ​behaviors”​ ​[1]​ ​and​ ​this acknowledgement​ ​will​ ​soon​ ​help​ ​us​ ​decide​ ​on​ ​our​ ​methods.​ ​Even​ ​in​ ​the​ ​worst​ ​case,​ ​under​ ​the assumption​ ​that​ ​playing​ ​penny-auctions​ ​is​ ​indeed​ ​probabilistic​ ​gambling​ ​[5],​ ​this​ ​research​ ​into building​ ​an​ ​agent​ ​to​ ​play​ ​the​ ​game​ ​could​ ​provide​ ​fascinating​ ​results​ ​as​ ​similar​ ​agents​ ​do​ ​with sports-betting​ ​[6,​ ​7]​ ​and​ ​blackjack​ ​[8].

Methodology

The​ ​first​ ​step​ ​is​ ​to​ ​collect​ ​as​ ​much​ ​data​ ​as​ ​possible​ ​in​ ​combination​ ​with​ ​existing​ ​datasets available​ ​about​ ​penny​ ​auction​ ​sites​ ​[1,​ ​9].​ ​This​ ​would​ ​involve​ ​building​ ​listeners​ ​to​ ​gather​ ​bids​ ​by users​ ​in​ ​a​ ​time​ ​series​ ​for​ ​each​ ​active​ ​auction​ ​on​ ​a​ ​site​ ​(along​ ​with​ ​its​ ​parameters).​ ​It’s​ ​also possible​ ​to​ ​negotiate​ ​licensing​ ​of​ ​a​ ​given​ ​sites’​ ​data​ ​with​ ​this​ ​permission,​ ​as​ ​the​ ​insights​ ​gleaned from​ ​an​ ​AI​ ​agent​ ​for​ ​their​ ​system​ ​could​ ​likely​ ​help​ ​company​ ​profitability​ ​in​ ​unexpected​ ​ways, especially​ ​for​ ​the​ ​companies​ ​with​ ​an​ ​illicit​ ​interest​ ​in​ ​shill​ ​bidding​ ​(they​ ​do​ ​exist)​ ​[10].

The​ ​next​ ​step​ ​would​ ​be​ ​training​ ​an​ ​agent​ ​based​ ​on​ ​the​ ​historical​ ​experiences​ ​of​ ​real​ ​life human​ ​agents.​ ​The​ ​data​ ​collected​ ​and​ ​its​ ​magnitude​ ​becomes​ ​extremely​ ​important​ ​here,​ ​and,​ ​if the​ ​strategy​ ​does​ ​not​ ​prove​ ​fruitful.​ ​I​ ​would​ ​focus​ ​my​ ​efforts​ ​on​ ​building​ ​instances​ ​of​ ​agents​ ​that map​ ​directly​ ​to​ ​actual​ ​human​ ​agents​ ​(by​ ​their​ ​usernames)​ ​that​ ​I​ ​could​ ​begin​ ​to​ ​use​ ​in​ ​multi-agent simulations.​ ​For​ ​the​ ​actual​ ​simulation​ ​itself,​ ​we’d​ ​build​ ​a​ ​simulated​ ​version​ ​of​ ​the​ ​penny-auction. Because​ ​there​ ​is​ ​a​ ​whole​ ​graveyard​ ​of​ ​now-defunct​ ​penny​ ​auction​ ​sites,​ ​acquiring​ ​the​ ​logical frameworks​ ​for​ ​these​ ​sites​ ​to​ ​pivot​ ​into​ ​a​ ​simulation​ ​is​ ​not​ ​out​ ​of​ ​the​ ​question.

With​ ​the​ ​success​ ​of​ ​one​ ​of​ ​these​ ​two​ ​methods,​ ​we​ ​would​ ​have​ ​the​ ​ability​ ​to​ ​do​ ​extensive controlled​ ​experimentation​ ​with​ ​regard​ ​to​ ​the​ ​economic​ ​theoretical​ ​implications​ ​of​ ​the​ ​“new” penny-auction​ ​game​ ​features,​ ​including​ ​“Buy-It-Now”,​ ​no-jumpers,​ ​bid​ ​packs,​ ​and​ ​fully symmetric​ ​player​ ​information​ ​and​ ​how​ ​these​ ​might​ ​affect​ ​the​ ​auctioneers​ ​profitability​ ​and player-type​ ​strategies.​ ​These​ ​results​ ​would​ ​be​ ​incredibly​ ​interesting​ ​to​ ​analyze​ ​and​ ​the framework​ ​would​ ​be​ ​quite​ ​powerful​ ​to​ ​put​ ​in​ ​the​ ​hands​ ​of​ ​fellow​ ​researchers,​ ​academics,​ ​and private​ ​interests.​ ​With​ ​full​ ​player​ ​information,​ ​one​ ​could​ ​do​ ​things​ ​in​ ​the​ ​vein​ ​of​ ​feasibly building​ ​out​ ​a​ ​fraud​ ​detection​ ​system​ ​that​ ​sniffs​ ​out​ ​shill​ ​bidders​ ​that​ ​are​ ​in​ ​employ​ ​of​ ​the auctioneers.​ ​The​ ​idea​ ​of​ ​q-learning​ ​or​ ​deep​ ​reinforcemnet​ ​learning​ ​agent​ ​playing​ ​penny-auction games​ ​is​ ​deeply​ ​interesting​ ​too,​ ​as​ ​it​ ​lies​ ​at​ ​the​ ​intersection​ ​of​ ​economic​ ​theory​ ​and​ ​computer science​ ​and​ ​seems​ ​to​ ​be​ ​a​ ​completely​ ​untapped​ ​research​ ​topic.​ ​I’d​ ​be​ ​excited​ ​to​ ​get​ ​feedback​ ​and hear​ ​thoughts​ ​from​ ​the​ ​market​ ​designers​ ​on​ ​this​ ​–​ ​especially​ ​regarding​ ​gleaning​ ​market​ ​insights from​ ​controlled​ ​simulations.

References

1. M.​ ​Mitzenmacher,​ ​John​ ​Byers,​ ​Georgios​ ​Zervas.​ ​Information​ ​Asymmetries​ ​in Pay-Per-Bid​ ​Auctions:​ ​How​ ​Swoopo​ ​Makes​ ​Bank.​ ​Unpublished​ ​manuscript.​ ​Available​ ​at http://people.bu.edu/zg/publications/BMZ10b-arxiv.pdf,​ ​2010.
2. T.​ ​Hinnosaar.​ ​Penny​ ​Auctions.​ ​Unpublished​ ​manuscript​ ​at​ ​http://toomas.hinnosaar.net/, 2009.
3. B.​ ​C.​ ​Platt,​ ​J.​ ​Price,​ ​and​ ​H.​ ​Tappen.​ ​Pay-to-Bid​ ​Auctions.​ ​Unpublished​ ​manuscript. Available​ ​at​ ​http://econ.byu.edu/Faculty/Platt,​ ​2009.
4. N.​ ​Augenblick.​ ​Consumer​ ​and​ ​Producer​ ​Behavior​ ​in​ ​the​ ​Market​ ​for​ ​Penny​ ​Auctions:​ ​A Theoretical​ ​and​ ​Empirical​ ​Analysis.​ ​Unpublished​ ​manuscript.​ ​Available​ ​at www.stanford.edu/​ ​~ned789,​ ​2009.
5. MacDonald,​ ​Colin​ ​Blake,​ ​”The​ ​Economics​ ​of​ ​Penny​ ​Auctions”​ ​(2010).​ ​Summer Research.​ ​Paper​ ​29.​ ​Available​ ​at​ ​http://soundideas.pugetsound.edu/summer_research/29, 2010.
6. Vincent,​ ​James.​ ​“This​ ​startup​ ​is​ ​building​ ​AI​ ​to​ ​bet​ ​on​ ​soccer​ ​games”.​ ​Available​ ​at https://www.theverge.com/2017/7/6/15923784/ai-predict-sport-betting-gambling-stratage m.​ ​2017
7. Sipko,​ ​Michal.​ ​“Machine​ ​Learning​ ​for​ ​the​ ​Prediction​ ​of​ ​Professional​ ​Tennis​ ​Matches”​ ​. Available​ ​at​ ​https://www.doc.ic.ac.uk/teaching/distinguished-projects/2015/m.sipko.pdf. 2015.
8. De​ ​Granville,​ ​Charles.​ ​“Applying​ ​Reinforcement​ ​Learning​ ​to​ ​Blackjack​ ​Using Q-Learning”.​ ​University​ ​of​ ​Oklahoma.​ ​Available​ ​at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.493.7116&rep=rep1&type=pdf
9. Swoopo​ ​Dataset.​ ​Available​ ​at​ ​http://cs-people.bu.edu/zg/swoopo-dataset.tar.gz.
10. Weisbaum,​ ​Herb.​ ​“Bidbots​ ​sometimes​ ​used​ ​to​ ​rig​ ​Internet​ ​penny​ ​auction​ ​sites”.​ ​Available at​ ​https://www.nbcnews.com/business/bidbots-sometimes-used-rig-internet-penny -auction-sites-1C8673443.​ ​2013.