Yes. It is possible to remove the agent's access to unassigned tickets views. In order to do accomplish this, you'll have to modify the agents' role (e.g. the Staff role) so that the setting What kind of tickets can this agent access? is set to Assigned to this agent only.
This prevents agents from being able to view or search for tickets that are not assigned to them. The agent will still be able to pull tickets into their inbox using the Playlist button despite not having access to them within the Zendesk user interface.
Here's a sample role with restricted access: