Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

init support for some reward handlers #86

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

EHAT32
Copy link
Contributor

@EHAT32 EHAT32 commented Feb 4, 2025

I am currently trying to implement reward handlers (RewardForSendingCommand and RewardForTouchingBlockType). I have stumbled upon a problem when rewards should be triggered on the same tick when agent quits the mission. For example, i am running cliff_walking example. Missions ends when agent touches the lava. Thus, mission ends but no reward for sending a command is triggered. The same thing with RewardForTouchingBlockType, at first agent touches the lava, mission ends and no reward for touching lava is triggered.
For the first implementation i uncommented this code in ClientStateMachine.MissionRunningEpisode.sendData() and modified getReward() a bit:

// Now create the reward signal:
if (currentMissionBehaviour() != null && currentMissionBehaviour().rewardProducer != null && cac != null)
            {
                MultidimensionalReward reward = new MultidimensionalReward();
                currentMissionBehaviour().rewardProducer.getReward(reward);
                if (!reward.isEmpty())
                {
                    String strReward = reward.getAsSimpleString();
//                    Minecraft.getMinecraft().mcProfiler.startSection("malmoSendTCPReward");

//                    ScoreHelper.logReward(strReward);

                    if (AddressHelper.getMissionControlPort() == 0) {
                        // MalmoEnvServer - reward
//                        if (envServer != null) {
//                            envServer.addRewards(reward.getRewardTotal());
//                        }

Also, i noticed there's a code for getting rewards with a message of ended mission

private void sendMissionEnded(MissionEnded missionEnded)
        {
            // Send a MissionEnded message to the agent to inform it that the mission has ended.
            // Create a string XML representation:
            String missionEndedString = null;
            try
            {
                missionEndedString = SchemaHelper.serialiseObject(missionEnded, MissionEnded.class);
                
                if (ScoreHelper.isScoring()) {
                    Reward reward = missionEnded.getReward();
                    if (reward == null) {
                        reward = new Reward();
                    }
                    ScoreHelper.logMissionEndRewards(reward);
                }
            }

I tried to uncomment it as well but at first glance it did nothing. Perhaps, MissionEnded class should be modified or re-implemented? If i got it right, it is a class from Malmo and needs some modifications to work with Vereya.

I would like to get some advices @CICS-Oleg @noskill

@EHAT32
Copy link
Contributor Author

EHAT32 commented Feb 4, 2025

After some further inspection i figured the problem might be in Python parsing MissionEndedXML and not in MissionEnded part in Java

@EHAT32 EHAT32 marked this pull request as ready for review February 8, 2025 09:23
@CICS-Oleg CICS-Oleg self-requested a review February 11, 2025 10:45
@noskill
Copy link
Collaborator

noskill commented Mar 3, 2025

In test_agent agent gets stuck looking down

image

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants