Is this evidence of some sort of pest I should worry about?

I've been noticing in one area of my garden/backyard/patio (whatever you want to call this tiny area I have here in Vancouver!) over the past few days that small piles of what could be droppings are turning up. This worries me as I have a little one who plays in the back yard sometimes. Here's the thing, though - I think it's too small for rat droppings, and it could be mouse droppings but none of the pictures I've seen on the web show them so small and piled up.

Here's what I know:

• We have had a Norway rat in the backyard before, but that was only when we tried a window birdfeeder a few years back. No bird feeder, no rat.
• There's nothing obvious that would harbour a family of mice -- there's a backyard cedar box, but everything a mouse would want is in plastic containers inside it, and I looked yesterday -- no mice. Otherwise it's stone pavers with Corsican mint & creeping thyme growing in between and two raised boxes, with nothing but sand under the fence right behind the boxes.
• If you get rid of the little pile, a new one will turn up the next day, slightly to the side of where it was before.
• But it only turns up around this one plant -- nothing anywhere else in the backyard.
• The little pile is slightly springy and not smelly or anything -- kind of like soil, but a bit springier, but not entirely like droppings, I think. But it's made of of little cylindrical things, kind of like droppings.
• We have seen plenty of slugs out back -- mostly when there was wood decking instead of pavers out back -- but I can't find a good picture of their droppings and I haven't seen an actual slug...plus it'd be weird that I can see these piles, they move a bit every day, and I can't see a slug anywhere.

The attached picture gives an idea -- note that the Corsican mint it's in is very small, and you're looking at a closeup of a 1" gap where the mint grows, so the little individual cylinders are much less than 1/4" long.

Anyone know what this could be? Is it a rat or a mouse? A worm of some kind?

 

Worm castings?

 

Thanks! That's what I was hoping, but just wanted to be sure as if they were mouse or rat droppings I wanted to make sure the little one stayed away.

One thing I forgot to mention is that I bury my bokashi compost in the raised box very close to this plant -- and there are lots of worms in the soil in the box there. I just wasn't sure if they'd go down into the sand & soil between the pavers.