<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<span style="margin: 0px; font-size: 12pt; color: black; background-color: white;">Prezados,</span>
<div style="margin: 0px; font-size: 12pt; color: black; background-color: white;">
<br>
</div>
<div style="margin: 0px; font-size: 12pt; color: black; background-color: white;">
Segue anuncio de vaga para doutorado em estatistica na Inglaterra, favor divulgar aos possiveis interessados.</div>
<div style="margin: 0px; font-size: 15px; font-family: "Segoe UI", "Segoe UI Web (West European)", "Segoe UI", -apple-system, BlinkMacSystemFont, Roboto, "Helvetica Neue", sans-serif; color: rgb(0, 0, 0); background-color: rgb(255, 255, 255);">
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black;">
<br>
</div>
<div style="margin:0px">
<div style="margin:0px">
<div style="margin:0px"></div>
<div style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black;">
Abracos,</div>
<span style="margin: 0px; font-size: 12pt; font-family: Calibri, Arial, Helvetica, sans-serif; color: black;">Cristine</span></div>
</div>
</div>
</div>
<div>
<div id="Signature">
<div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
</div>
</div>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> A UK-based worldwide e-mail broadcast system mailing list <allstat@JISCMAIL.AC.UK> on behalf of Leslie, David <d.leslie@LANCASTER.AC.UK><br>
<b>Sent:</b> 02 March 2022 12:06 PM<br>
<b>To:</b> allstat@JISCMAIL.AC.UK <allstat@JISCMAIL.AC.UK><br>
<b>Subject:</b> [External] PhD position, Lancaster University, with international funding</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">This email originated outside the University. Check before clicking links or attachments.<br>
<br>
Topic: Reinforcement learning with structured action spaces<br>
Supervisors: David Leslie (Lancaster) and Raphael Clifford (Bristol)<br>
Closing date: When we have found a suitable candidate<br>
Start date: October 2022<br>
<br>
Reinforcement learning (RL) is machine learning technique in which computers experiment with an environment and learn effective behaviour. The problems in which RL are particularly effective are sequential decision-making problems where the task to be solved
consists of observing the state of the environment, selecting an action, incurring some cost, and moving to a new state, where the cost and the successor state depend both on the initial state and the action selected; the canonical mathematical formulation
of these challenges is a Markov decision process. The most famous recent example of RL success is the game of Go, addressed by Deepmind, which builds on a rich history in both games and individual decision-making examples (see Sutton and Barto (2018) for a
survey).<br>
<br>
In common reinforcement learning approaches, the set of actions which is available to the decision maker at each time instant is very regular. In many examples it is either fixed, finite and small (e.g. move North, South, East or West), or a simple continuous
space (an angle and speed to move at). However in lots of problems, the action space is more complex. In robotic soccer-playing environments, the player can choose whether to run, turn or kick, and each of these choices is then parameterised by the strength
and/or direction; this type of action space, with a finite number of action families each of which is indexed by a parameter, is called a parameterised action space.<br>
<br>
In contrast to image processing, for which standard deep learning methods and libraries now exist, when action sets with complex structure are encountered, custom solutions have generally been required. This custom approach severely hinders the ability of non-specialists
to deploy RL methods on their own problems. Thus the focus of the PhD topic is to formulate and code modular reinforcement learning components for general structured action spaces.<br>
The project could take several directions, including:<br>
• devising policy optimisation analogues of existing value learning approaches for structured action spaces<br>
• extending parameterised action space approaches to more general structured action spaces<br>
• deriving exploration strategies for parameterised action spaces to ensure efficient experimentation<br>
A successful candidate will have skills in both mathematics and computer science - you will formulate methods for awkward action spaces, implement methods in modular code, and run computer experiments to compare methods on various problems.<br>
<br>
To start the application process, please send your undergraduate transcript to d.leslie@lancaster.ac.uk with a brief note about why this project interests you.<br>
<br>
--<br>
<br>
David Leslie (he/him/his), Professor of Statistical Learning,<br>
Head of Statistics, Lancaster University<br>
<br>
<br>
<br>
<br>
<br>
You may leave the list at any time by sending the command<br>
<br>
SIGNOFF allstat<br>
<br>
to listserv@jiscmail.ac.uk, leaving the subject line blank.<br>
</div>
</span></font></div>
</body>
</html>