How to know if an RL agent has been updated
Hi all,
I want to train an RL agent, but would like to make sure that my agent is updated, so I want to ask how to see if the agent has been updated.
For example, in the official example of ‘rl/TrainMultipleAgentsForAreaCoverageExample’, I extracted the code related to the agent definition, the training and the simulation:
%…
agentA = rlPPOAgent(actor(1),critic(1),opt);
agentB = rlPPOAgent(actor(2),critic(2),opt);
agentC = rlPPOAgent(actor(3),critic(3),opt);
%…
if doTraining
result = train([agentA,agentB,agentC],env,trainOpts);
else
load("rlAreaCoverageAgents.mat");
end
%…
rng(0) % reset the random seed
simOpts = rlSimulationOptions(MaxSteps=maxsteps);
experience = sim(env,[agentA,agentB,agentC],simOpts);
However, say after training I would like to do a check on whether the agentA has changed or not:
copy = agentA;
%the above code section where agentA is trained…
disp(copy==agentA)
The result displayed is 1, so agentA has not been changed?
But this is from the official example so I believe the agents should indeed have been trained. And the simulation result also suggests that they have been trained since it takes sufficiently longer for an agent before train() to complete the task than the one after train().
It seems that train() does update agents, but how can I explicitly tell from the variables in my workspace that they are indeed updated? And why he above comparison is not working? Thank you.
Haochen TaoHi all,
I want to train an RL agent, but would like to make sure that my agent is updated, so I want to ask how to see if the agent has been updated.
For example, in the official example of ‘rl/TrainMultipleAgentsForAreaCoverageExample’, I extracted the code related to the agent definition, the training and the simulation:
%…
agentA = rlPPOAgent(actor(1),critic(1),opt);
agentB = rlPPOAgent(actor(2),critic(2),opt);
agentC = rlPPOAgent(actor(3),critic(3),opt);
%…
if doTraining
result = train([agentA,agentB,agentC],env,trainOpts);
else
load("rlAreaCoverageAgents.mat");
end
%…
rng(0) % reset the random seed
simOpts = rlSimulationOptions(MaxSteps=maxsteps);
experience = sim(env,[agentA,agentB,agentC],simOpts);
However, say after training I would like to do a check on whether the agentA has changed or not:
copy = agentA;
%the above code section where agentA is trained…
disp(copy==agentA)
The result displayed is 1, so agentA has not been changed?
But this is from the official example so I believe the agents should indeed have been trained. And the simulation result also suggests that they have been trained since it takes sufficiently longer for an agent before train() to complete the task than the one after train().
It seems that train() does update agents, but how can I explicitly tell from the variables in my workspace that they are indeed updated? And why he above comparison is not working? Thank you.
Haochen Tao Hi all,
I want to train an RL agent, but would like to make sure that my agent is updated, so I want to ask how to see if the agent has been updated.
For example, in the official example of ‘rl/TrainMultipleAgentsForAreaCoverageExample’, I extracted the code related to the agent definition, the training and the simulation:
%…
agentA = rlPPOAgent(actor(1),critic(1),opt);
agentB = rlPPOAgent(actor(2),critic(2),opt);
agentC = rlPPOAgent(actor(3),critic(3),opt);
%…
if doTraining
result = train([agentA,agentB,agentC],env,trainOpts);
else
load("rlAreaCoverageAgents.mat");
end
%…
rng(0) % reset the random seed
simOpts = rlSimulationOptions(MaxSteps=maxsteps);
experience = sim(env,[agentA,agentB,agentC],simOpts);
However, say after training I would like to do a check on whether the agentA has changed or not:
copy = agentA;
%the above code section where agentA is trained…
disp(copy==agentA)
The result displayed is 1, so agentA has not been changed?
But this is from the official example so I believe the agents should indeed have been trained. And the simulation result also suggests that they have been trained since it takes sufficiently longer for an agent before train() to complete the task than the one after train().
It seems that train() does update agents, but how can I explicitly tell from the variables in my workspace that they are indeed updated? And why he above comparison is not working? Thank you.
Haochen Tao reinforcement learning, agent MATLAB Answers — New Questions