While I have a site with 40k pages and 40 sitemaps files with 1k urls each (1 line, 1 url) and getting 50% index ratio and basically having the same situation as Rob when I had 1 sitemap file – I’m still wondering how I could get more like 100% – The customer should have the possibility of finding the right page…
The site is very much so an ecosystem in its own right and thus ever changing.
Using more (and ever more) sitemaps solved – rather helped – the 8000/30000 index ratio situation… buuuut, it’s a little lame… even if it works, its a lot of extra work.
I’m pretty sure internal linking could not really be the “baddy” here – ’cause all pages are linked to from the top and all pages links to all pages in the same group through “list views” of those groups – as I’m sure Robs dynamic 30k pages site is too in some way or another… Rob? What site are you referring to?
I would probably never use the XML solution, as I too is in great need of another reason.. =)

Not 100% sure what you’re agreeing/disagreeing with. I was merely pointing out that by submitting an XML Sitemap to Google, you could view the ratio of pages listed in your sitemap to those actual indexed (as reported by Google).

It may very well be possible that your 30K page site only has 8000 indexed. While I’m a proponent of using and submitting sitemaps, I’ve never been one to naively believe that it’s a ticket to 100% indexation! You still have to work on the fundamentals of internal linking and site structure to ensure that Google, and the rest of the search engines, can find your content and index/rank it appropriately!

Thanks for commenting!