I've seen the sql.Scanner and driver.Valuer interfaces re-implemented twice now, once in Gorp and again in meddler. Since Go's otherwise excellent documentation lacks some of the more user-story-centric aspects of non source-based docs, I thought this might be because it's unclear that these interfaces exist and can be used to provide this functionality.

Let's take a look at the definitions of these interfaces:

type Valuer interface {
        // Value returns a driver Value.
        Value() (Value, error)
}

Valuer is a simple interface; a function which must return a driver.Value (which is interface{}, but in practice it should be a set of the restricted types listed in the Scanner interface below) and optionally an error if there was a problem. This interface is used by database/sql when binding variables in query strings for query execution.

type Scanner interface {
        // Scan assigns a value from a database driver.
        //
        // The src value will be of one of the following restricted
        // set of types:
        //
        //    int64, float64, , bool, []byte, string, time.Time
        //    nil - for NULL values
        //
        // An error should be returned if the value can not be stored
        // without loss of information.
        Scan(src interface{}) error
}

Scanner is a slightly more complex interface to use, because you must mutate or reassign the variable being scanned into, and you'll receive data which is of an unknown type. This interface is used by the Scan function to to assign values to custom types, like sql.NullString.

Let's go over some simple examples on creating custom types which use these interfaces to achieve two things we might want to do on insert and retrieval: validation and serialization.

First, imagine that we wanted to limit phone number storage to one awful standard, (###) ###-###, which we'll implement as a check against the regex \(\d{3}\) \d{3}-\d{4}. We can establish this check directly on the type we will store in the database by implementing the Valuer interface:

type PhoneNumber string

var phoneNumberCheck = regexp.MustCompile(`\(\d{3}\) \d{3}-\d{4}`)

func (p PhoneNumber) Value() (driver.Value, error) {
    matched := phoneNumberCheck.Match([]byte(p))
    if !matched {
        return driver.Value(""), fmt.Errorf("Number '%s' not a valid PhoneNumber format.", p)
    }
    return driver.Value(string(p)), nil
}

This will now check PhoneNumbers on their way to the database when the data is bound:

db.Exec("CREATE TABLE IF NOT EXISTS person ( name text, address text, phonenumber text);")
query := `INSERT INTO person
    (name, address, phonenumber)
        VALUES
    (?, ?, ?)`
// this works
_, err = db.Exec(query, "Ben", "Smith", PhoneNumber("(123) 456-0987"))
// this returns an error!
_, err = db.Exec(query, "Ma'a", "Nonu", PhoneNumber("123.456.7890"))
fmt.Println("Error in exec: ", err)
// Error in exec:  sql: converting Exec argument #2's type: Number '123.456.7890' not a valid PhoneNumber format.

Let's take a look at a more complex example now, which uses both interfaces. Suppose we wanted to have a field stored as gzipped compressed text in the database. We'll want to implement the Valuer interface, as before, as a way of automatically compressing data on its way to the database:

type GzippedText []byte

func (g GzippedText) Value() (driver.Value, error) {
    b := make([]byte, 0, len(g))
    buf := bytes.NewBuffer(b)
    w := gzip.NewWriter(buf)
    w.Write(g)
    w.Close()
    return buf.Bytes(), nil
}

This time, we'll also implement Scanner, to decompress data coming from the database:

func (g *GzippedText) Scan(src interface{}) error {
    var source []byte
    // let's support string and []byte
    switch src.(type) {
    case string:
        source = []byte(src.(string))
    case []byte:
        source = src.([]byte)
    default:
        return errors.New("Incompatible type for GzippedText")
    }
    reader, _ := gzip.NewReader(bytes.NewReader(source))
    defer reader.Close()
    b, err := ioutil.ReadAll(reader)
    if err != nil {
        return err
    }
    *g = GzippedText(b)
    return nil
}

Our GzippedText type can now be used as easily as a []byte is!

db.Exec("CREATE TABLE IF NOT EXISTS compressed (content text);")
db.Exec("INSERT INTO compressed (content) VALUES (?)", GzippedText("Hello, world."))
row := db.QueryRow("SELECT * FROM compressed LIMIT 1;")
var raw []byte
var unz GzippedText
row.Scan(&raw)
fmt.Println("Raw:", raw)
// Raw: [31 139 8 0 0 9 110 136 0 255 242 72 205 201 201 215 81 40 207 47 202 73 209 3 4 0 0 255 255 119 219 89 123 13 0 0 0]
row = db.QueryRow("SELECT * FROM compressed LIMIT 1;")
row.Scan(&unz)
fmt.Println("Unzipped:", string(unz))
// Unzipped: Hello, world.

These types are usable anywhere that database/sql is used to bind something, which includes ormish libraries like gorp and modl, sql extensions like meddler and sqlx, as well as any future libraries which use or extend that interface. The GzippedText interface can be used anywhere Scan is used, including as fields in sqlx StructScan destinations; as an exercise, extend the PhoneNumber type to do the same. You can view this code in a form that will build on GitHub.

Many built-in Go interfaces are wonderfully extensible, and the re-implementation of parts of these interfaces isn't limited to the SQL library. There are a number of middleware implementations out there in various microframeworks which do not rely on the capable http.Handler interface and are thus not usable across different libraries. The same is true of abstracting away the ServeHTTP with some other wrapper; as reddit user sjustinas remarked on a microframework discussion:

[...] I think you hide the original ResponseWriter. Don't. Go has a beautiful standard HTTP interface and even if you abstract it away, leave the user a way to access it in case it's needed.

For instance, if your handler function can only return a []byte or a string, you take away any streaming functionality.

Edit: They've followed up their comment with a nice blog post which mirrors a lot of the sentiments in this post.

One of the strengths of Go is that you can replace "standard HTTP interface" with many of its other standard interfaces and much of the sentiment in the above quote would remain accurate.

Oct 14 2013