I just changed the oil on this drive that has 25 hours on it.

It blew up last August, and was rebuilt by a friend. That rebuild last 30 minutes. Got rebuilt by Joey Griffin in NC. Has been run 25 hours at mostly 3500 rpms. This is first oil change. I am thinking (hoping) the fuzz and chunks are leftovers from the 2 previous times it blew up and/or it is from break in.
Joey Griffin put in new gears, upper drive shaft, bearings and not sure what else.

I'll check the oil after a run or two, but an interested in the collective wisdom here - should I be worried?

Nope actually its quite normal considering it has new gears.
With new gears you will get a small amount of metal fuzz and as its used over time just a normal thing. Just make sure its not huge amounts or chunks.

There were chunks in there.

okay so I guess the question is what do you consider a chunk? how big? any pics?
I would say you might find small debris, nothing that big though

That looks like one of mine did this spring.

It had a pinion gear and the lower gear ( in the upper)
chunking out.

I'd be tearing it apart.

I'm with JT. Take it apart and inspect it. If nothing wrong, no problem. If something IS WRONG, then you catch it now, not later at the expense of more gears, bearings, etc.

That is not just fuzz. I would take it apart.

I had one like that last fall, tore it down and the pinion nut had backed off, somethings not right.

Bill,
If you can feel any chunks, Take It Apart!! It will be far better to find it now than later!
The fuzz on a magnet from new gears should be like pencil lead and no more. Any flakes or chunks and gears are coming apart..
Hope this helps.
Dick