Sparse Coding

Sparse coding is the development of a distributed binary representation of arbitrary entities. It needs two requirements: (i) sparseness means that there are many more 0s than 1s in the codewords. (ii) Given a notion of similarity or dissimilarity on the set of entities, for example a distance or metric, the code should be similarity preserving, i.e., similar entities should set similar codewords (with respect to the Hamming distance). We are working on the development of such codes in various applications (e.g., words, images, speech). They are particularly useful for associative memories.

Günther Palm