We consider applications where agents have to cooperate without any communication taking place between them, apart from the fact that they can see part of the environment in which they act. We present a multi-agent system, defined in Golog, that needs to service tasks whose value degrades in time. Initial plans, reflecting prior knowledge about the environment, are expressed as Golog procedures, and are provided to the agents. Then the agents are trained using reinforcement learning, in order to ensure coordination both at the action level and at the plan level. This ensures better scalability and increased performance of the system.