Writing a software renderer is quickly associated with revolutionizing things, but on the more conservative side, one advantage would really be that one has the freedom to optimize things as needed rather than having to try and guess what a driver is doing behind the scenes (maybe write your own driver ?!).
Graphics drivers do a great deal of work, they can make a big differences, they can be quite smart at guessing resources usage, what to prioritize on which basis, but they can't be smarter than a whole application.

APIs such as Direct3D and OpenGL don't have a concept of object (ok, OpenGL supports "lists" at least). So, they miss potentially useful hints. For example, if you know your object bounding box, you can tell right away whether or not the object requires clipping and you can tell in advance which textures you need, and what's the maximum level of detail that is needed for textures (if the bounding box is 512x512, you can't possibly need a 1024x1024 texture 8).
Based on that knowledge, one could completely avoid having to perform per-polygon clipping test and could also avoid loading full size textures in video RAM.. because a smaller mip-level would be sufficient.

Another big problem in dealing with a separate hardware graphics system is communication. Every state change can be a big deal. Drivers will probably cache things, but not necessarily.
Recently, I wrote an immediate rendering library to draw debug primitives. I had a small pool of vertex buffers that I would rotate as I called Draw() several times per frame. It turned out to be a big slowdown, so big that I had to switch to use one vertex buffer per frame (actually 2/3 to rotate at each frame, not at each Draw()).
To do that of course I had to keep track of logical draw calls issued by the application program, so that I could finally unmap/unlock the vertex buffer at the end and call all the Draw() at once.. remembering of which primitive type it was, how many vertices and which draw state was associated with that draw call.

This all comes down to having to deal with separate architectures. I have my vertex buffer, the card has its vertex buffer.. collect here, copy there, avoid touching this buffer or that buffer.
Small things that show the cat and mouse kind of job of having to optimize rendering using an API such as Direct3D 10.

Lastly, recently I managed to crash nVidia's driver on Vista 8)
I think it has to do with.. big polygons, or polygons going way off.. possibly a clipping bug ? All I know is that my screen goes blank and boom !
The window comes back and it's all black, while I get a balloon message from the sys-tray that says that the driver's process has crashed. From then on I can't do 3D unless I reboot 8)

ehhhhhhhhhhh