How do you assess the quality of a remix album? The tracks are always in reference to something else, and yet because of the stamp of authority that the collaborating artists wants to put on their interpretation of the song, the tracks are sufficiently different from their source that they should maybe merit being thought of as original works. Should Radiohead's TKOL RMX 1234567 be thought of as just another album, then?

It's certainly worth asking whether it's up to the task: with contributors including Four Tet, Modeselektor, Caribou, Jamie xx, SBTRKT, Shed, Lone, and Nathan Fake, the two-disc compilation isn't without talent. Unfortunately, and especially at 19 tracks and over one hour of listening time, it's an album short of quality, assessed on its own merits and also, in parts, in relation to the catalogues of the contributing artists. Nothing really stands out, while nothing is particularly bad either, and it doesn't really feel like an album you'd need or want to listen to again and again.

But this album isn't really made to stand alone. Produced with existing fans in mind, it is in theory a re-augmentation of a body of work that will be recognised; as hard as you might try, there's no getting away from the fact that everything on TKOL RMX is made in reference to and in dialogue with The King of Limbs. And that's the problem with a remixed work: it's always stuck in the difficult no-man's land that is having an expectation that it will be a new work, but that it also references sounds which you know and love.

This is a particularly acute feeling with the two disc TKOL RMX album, because Radiohead fans love Radiohead so intently. They're a hard bunch to please – are often evangelical, often idolatry. It's a funny fact, because an important reason that Radiohead have such a devout following, critically as well as in terms of a general audience, is because they are perceived to “reinvent” themselves, even as supreme fame rears its head: their famous response to the incredible success of OK Computer was to produce the anti-pop Kid A, and The King of Limbs, which has saddened some critics and fans, did something similar, in its own way, in response to In Rainbows.

They are a band whose catalogue can, at times, sound radically different: a set of remixes theoretically shouldn't have anybody up in arms, because it's just part of that process of musical creation, of progression.

And yet, TKOL RMX seems curiously...lacking. The reason for this, as it becomes clear only a few tracks in, is because Thom Yorke's voice is one of the most recognisable on the planet; it's easy to see why he wanted to hide it so much on Kid A. And these remixes, because they deal with what is effectively a pop formula (Radiohead complicate the genre, but it's still broadly speaking popular, tonal music) have nearly all succumbed to the temptation to play around with the vocal line, mess up Yorke's voice in some way. Caribou's remix of 'Little by Little' for example, pitch shifts the vocal line on the chorus, to depressing effect (Modesleketor's 'Good Evening Mr Magpie' does it on the verses), and Nathan Fake's 'Morning Mr. Magpie' (as well as, to some extent, the Illum Sphere 'Codex' remix) chops it until it sounds like the original vocal line skipping over a chilled out club track.

Which is one of the other problems here: because of the backgrounds of a number of the artists involved, most of the tracks rely on electro and dance tropes. Sometimes that works – Four Tet's version of 'Separator' is very good – but it loses something about Radiohead that's always been impressive: their ability to circle around tone and emotion without centring in, in a proper kind of ambivalence.

TKOL RMX really suffers from that loss. Caribou's contribution for example, as well as Jacques Greene's 'Lotus Flower' (which it has to be said, works very well) are upbeat, even happy, while the originals fizz with a cluster of emotions over every beat. While the remixed tracks themselves aren't bad (if you want to imagine them as tracks in their own right) they're just not that good either,and that element of the music is certainly lost.

It's important in that regard to consider whether the album counts as a Radiohead album. Certainly, it uses their music, but is it a Radiohead album without that ambivalence? And should it be assessed differently, (perhaps more favourably) according to different criteria, if it isn't?

Maybe the answer lies in the medium. The 19 tracks compiled on TKOL RMX 1234567 have first been released across seven singles (with the last due on 10 October 2011) so that the tracks are not only split up, but also sometimes loosely themed; the first, for example, contained two Caribou remixes and the Jacques Green 'Lotus Flower' remix, all of which exhibit tonal similarities. Released in this way, the tracks feel more like an experiment, a set of things that can pique your interest, and the length of each single is short enough that it doesn't wear out. It also means that each single can exist as a set of “new” compositions: it doesn't need to feel like a Radiohead album, and any expectations you might have about what a Radiohead album might be can stay comfortably out of things.

Compiled in 19 tracks, though, that isn't such an easy thing. And in fact, what these remixes do is make you realise just what makes Radiohead, a band so often revered for continually changing their sound, so appealing: their incredibly strong sense of personality seeps through their work. The voice and vocal lines of Yorke have already been discussed, but Jonny Greenwood, as influential as Yorke on alternative music of the last two decades, plays in a way you can really pick out; you can tell what belongs to him, just as you can normally spot the contribution of Ed O'Brien, and you can hear Colin Greenwood's bass, perhaps the most singularly common aspect of their sound, bobbing away.

Their sense of identity is incredibly strong, and to their continuing success: despite the differences between The King of Limbs and, say, The Bends you can still tell its the same band. There is a personality there; an ambitious, interesting and satisfying personality.

But these tracks don't have that. They lose that sense of personality, even if what they gain is a set of interesting sounds. Thriller Houseghost's remix of 'Give up the Ghost' is a really good example of that. Its take on the track, to up the tempo slightly (also to electrify it) is an interesting idea, and its main loops are sweet, soulful and sound; but as a remix of a very Yorke-orientated track, it doesn't ever really get off the ground, and peters out quite quickly.

There is of course, another element to TKOL RMX, in that a lot of the music that is referencing Radiohead, Radiohead once referenced; it's hard to imagine that the musical approach of Four Tet and others didn't have something of an influence on their work with Amnesiac, to some extent Kid A, and certainly Hail to the Thief. It's interesting that these acts, seen to be paying tribute to Radiohead, are in some ways direct descendants of music that Radiohead have themselves appropriated, paid tribute to, made their own.

It's a shame that none of the remixes take the same approach. A version of 'Codex' that stripped everything but the incredibly strong vocal line and the synth-strings could be sublime, even if just a little more were added around it. But every track seems to want to be as creative as it can, rather than just to re-mix. There has to be, it seems, a transformation, rather than a development.

That, ultimately, is what stops TKOL RMX 1234567 from working particularly well. At best, it works as a thought experiment rather than an album; it makes you realise how much music is a set of contingent choices, and how one melody (usually a vocal hook in the case of these remixes) can give rise to any number of variations. It's an experiment in what Radiohead could sound like but don't. But as a set of tracks, it isn't really all that much to write home about.