I am not a techie but i think an amp do improve the sound if your source is not up to it.
I just bought a Fiio e9 and it improved the sound on my mp3 player a lot with cleaner, sharper sound and tighter punchier bass. Before everything sounded a little off. But for my computer, the difference was minimal.
I dunno buying something small and cheap like the e5. Considering that my e9 only made what I consider marginal improvements I would expect the e5 to make much much less that I probably won't notice the difference.