In the third and final approach of uploading a picture, we will look at how to allow users to upload an image from their local hard drive to use as their profile picture when chatting. We will need a way to associate a file with a particular user to ensure that we associate the right picture with the corresponding messages.
In order to uniquely identify our users, we are going to copy Gravatar's approach by hashing their e-mail address and using the resulting string as an identifier. We will store the user ID in the cookie along with the rest of the user-specific data. This will actually have the added benefit of removing from GravatarAuth
the inefficiency associated with continuous hashing.
In auth.go
, replace the code that creates the authCookieValue
object with the following code:
m := md5.New() io.WriteString(m, strings.ToLower(user.Name())) userId := fmt.Sprintf("%x", m.Sum(nil)) // save some data authCookieValue := objx.New(map[string]interface{}{ "userid": userId, "name": user.Name(), "avatar_url": user.AvatarURL(), "email": user.Email(), }).MustBase64()
Here we have hashed the e-mail address and stored the resulting value in the userid
field at the point at which the user logs in. Henceforth, we can use this value in our Gravatar code instead of hashing the e-mail address for every message. To do this, first we update the test by removing the following line from avatar_test.go
:
client.userData = map[string]interface{}{"email": "[email protected]"}
We then replace the preceding line with this line:
client.userData = map[string]interface{}{"userid": "0bc83cb571cd1c50ba6f3e8a78ef1346"}
We no longer need to set the email
field since it is not used; instead, we just have to set an appropriate value to the new userid
field. However, if you run go test
in a terminal, you will see this test fail.
To make the test pass, in avatar.go
, update the GetAvatarURL
method for the GravatarAuth
type:
func (_ GravatarAvatar) GetAvatarURL(c *client) (string, error) { if userid, ok := c.userData["userid"]; ok { if useridStr, ok := userid.(string); ok { return "//www.gravatar.com/avatar/" + useridStr, nil } } return "", ErrNoAvatarURL }
This won't change the behavior, but it allows us to make an unexpected optimization, which is a great example of why you shouldn't optimize code too early—the inefficiencies that you spot early on may not last long enough to warrant the effort required to fix them.
If our users are to upload a file as their avatar, they need a way to browse their local hard drive and submit the file to the server. We facilitate this by adding a new template-driven page. In the chat/templates
folder, create a file called upload.html
:
<html> <head> <title>Upload</title> <link rel="stylesheet" href="//netdna.bootstrapcdn.com/bootstrap/3.1.1/css/bootstrap.min.css"> </head> <body> <div class="container"> <div class="page-header"> <h1>Upload picture</h1> </div> <form role="form" action="/uploader" enctype="multipart/form-data" method="post"> <input type="hidden" name="userid" value="{{.UserData.userid}}" /> <div class="form-group"> <label for="message">Select file</label> <input type="file" name="avatarFile" /> </div> <input type="submit" value="Upload" class="btn " /> </form> </div> </body> </html>
We used Bootstrap again to make our page look nice and also to make it fit in with the other pages. However, the key point to note here is the HTML form that will provide the user interface necessary for uploading files. The action points to /uploader
, the handler for which we have yet to implement, and the enctype
attribute must be multipart/form-data
so the browser can transmit binary data over HTTP. Then, there is an input
element of the type file
, which will contain the reference to the file we want to upload. Notice also that we have included the userid
value from the UserData
map as a hidden input—this will tell us which user is uploading a file. It is important that the name
attributes are correct, as this is how we will refer to the data when we implement our handler on the server.
Let's now map the new template to the /upload
path in main.go
:
http.Handle("/upload", &templateHandler{filename: "upload.html"})
When the user clicks on Upload after selecting a file, the browser will send the data for the file as well as the user ID to /uploader
, but right now, that data doesn't actually go anywhere. We will implement a new HandlerFunc
that is capable of receiving the file, reading the bytes that are streamed through the connection, and saving it as a new file on the server. In the chat
folder, let's create a new folder called avatars
—this is where we will save the avatar image files.
Next, create a new file called upload.go
and insert the following code—make sure to add the appropriate package name and imports (which are ioutils
, net/http
, io
, and path
):
func uploaderHandler(w http.ResponseWriter, req *http.Request) { userId := req.FormValue("userid") file, header, err := req.FormFile("avatarFile") if err != nil { io.WriteString(w, err.Error()) return } data, err := ioutil.ReadAll(file) if err != nil { io.WriteString(w, err.Error()) return } filename := path.Join("avatars", userId+path.Ext(header.Filename)) err = ioutil.WriteFile(filename, data, 0777) if err != nil { io.WriteString(w, err.Error()) return } io.WriteString(w, "Successful") }
Here, first uploaderHandler
uses the FormValue
method on http.Request
to get the user ID that we placed in the hidden input in our HTML form. Then it gets an io.Reader
type capable of reading the uploaded bytes by calling req.FormFile
, which returns three arguments. The first argument represents the file itself with the multipart.File
interface type, which is also an io.Reader
. The second is a multipart.FileHeader
object that contains metadata about the file, such as the filename. And finally, the third argument is an error that we hope will have a nil
value.
What do we mean when we say that the multipart.File
interface type is also an io.Reader
? Well, a quick glance at the documentation at http://golang.org/pkg/mime/multipart/#File makes it clear that the type is actually just a wrapper interface for a few other more general interfaces. This means that a multipart.File
type can be passed to methods that require io.Reader
, since any object that implements multipart.File
must therefore implement io.Reader
.
Embedding standard library interfaces to describe new concepts is a great way to make sure your code works in as many contexts as possible. Similarly, you should try to write code that uses the simplest interface type you can find, ideally from the standard library. For example, if you wrote a method that needed to read the contents of a file, you could ask the user to provide an argument of the type multipart.File
. However, if you ask for io.Reader
instead, your code will become significantly more flexible because any type that has the appropriate Read
method can be passed in, which includes user-defined types too.
The ioutil.ReadAll
method will just keep reading from the specified io.Reader
until all of the bytes have been received, so this is where we actually receive the stream of bytes from the client. We then use path.Join
and path.Ext
to build a new filename using userid
, and copy the extension from the original filename that we can get from multipart.FileHeader
.
We then use the ioutil.WriteFile
method to create a new file in the avatars
folder. We use userid
in the filename to associate the image with the correct user, much in the same way as Gravatar does. The 0777
value specifies that the new file we create has full file permissions, which is a good default setting if you're not sure what other permissions should be set.
If an error occurs at any stage, our code will write it out to the response, which will help us debug it, or it will write Successful if everything went well.
In order to map this new handler function to /uploader
, we need to head back to main.go
and add the following line to func main
:
http.HandleFunc("/uploader", uploaderHandler)
Now build and run the application and remember to log out and log back in again to give our code a chance to upload the auth
cookie.
go build -o chat ./chat -host=:8080
Open http://localhost:8080/upload
and click on Choose File, then select a file from your hard drive and click on Upload. Navigate to your chat/avatars
folder and you will notice that the file was indeed uploaded and renamed to the value of your userid
field.
Now that we have a place to keep our users' avatar images on the server, we need a way to make them accessible to the browser. We do this by using the net/http
package's built-in file server. In main.go
, add the following code:
http.Handle("/avatars/", http.StripPrefix("/avatars/", http.FileServer(http.Dir("./avatars"))))
This is actually a single line of code that has been broken up to improve readability. The http.Handle
call should feel familiar: we are specifying that we want to map the /avatars/
path with the specified handler—this is where things get interesting. Both http.StripPrefix
and http.FileServer
return Handler
, and they make use of the decorator pattern we learned about in the previous chapter. The StripPrefix
function takes Handler
in, modifies the path by removing the specified prefix, and passes functionality onto an inner handler. In our case, the inner handler is an http.FileServer
handler that will simply serve static files, provide index listings, and generate the 404 Not Found
error if it cannot find the file. The http.Dir
function allows us to specify which folder we want to expose publicly.
If we didn't strip the /avatars/
prefix from the requests with http.StripPrefix
, the file server would look for another folder called avatars
inside the actual avatars
folder, that is, /avatars/avatars/filename
instead of /avatars/filename
.
Let's build the program and run it before opening http://localhost:8080/avatars/
in a browser. You'll notice that the file server has generated a listing of the files inside our avatars
folder. Clicking on a file will either download the file, or in the case of an image, simply display it. If you haven't done so already, go to http://localhost:8080/upload
and upload a picture, then head back to the listing page and click on it to see it in the browser.
The final piece to making filesystem avatars work is to write an implementation of our Avatar
interface that generates URLs that point to the filesystem endpoint we created in the last section.
Let's add a test function to our avatar_test.go
file:
func TestFileSystemAvatar(t *testing.T) { // make a test avatar file filename := path.Join("avatars", "abc.jpg") ioutil.WriteFile(filename, []byte{}, 0777) defer func() { os.Remove(filename) }() var fileSystemAvatar FileSystemAvatar client := new(client) client.userData = map[string]interface{}{"userid": "abc"} url, err := fileSystemAvatar.GetAvatarURL(client) if err != nil { t.Error("FileSystemAvatar.GetAvatarURL should not return an error") } if url != "/avatars/abc.jpg" { t.Errorf("FileSystemAvatar.GetAvatarURL wrongly returned %s", url) } }
This test is similar to, but slightly more involved than, the GravatarAvatar
test because we are also creating a test file in our avatars
folder and deleting it afterwards.
The rest of the test is simple: we set a userid
field in client.userData
and call GetAvatarURL
to ensure we get back the right value. Of course, running this test will fail, so let's go and add the following code to make it pass in avatar.go
:
type FileSystemAvatar struct{} var UseFileSystemAvatar FileSystemAvatar func (_ FileSystemAvatar) GetAvatarURL(c *client) (string, error) { if userid, ok := c.userData["userid"]; ok { if useridStr, ok := userid.(string); ok { return "/avatars/" + useridStr + ".jpg", nil } } return "", ErrNoAvatarURL }
As we see here, to generate the correct URL, we simply get the userid
value and build the final string by adding the appropriate segments together. You may have noticed that we have hardcoded the file extension to .jpg
, which means that the initial version of our chat application will only support JPEGs.
Let's see our new code in action by updating main.go
to use our new Avatar
implementation:
r := newRoom(UseFileSystemAvatar)
Now build and run the application as usual and go to http://localhost:8080/upload
and use a web form to upload a JPEG image to use as your profile picture. To make sure it's working correctly, choose a unique image that isn't your Gravatar picture or the image from the authentication service. Once you see the successful message after clicking on Upload, go to http://localhost:8080/chat
and post a message. You will notice that the application has indeed used the profile picture that you uploaded.
To change your profile picture, go back to the /upload
page and upload a different picture, then jump back to the /chat
page and post more messages.
To support different file types, we have to make our GetAvatarURL
method for the FileSystemAvatar
type a little smarter.
Instead of just blindly building the string, we will use the very useful ioutil.ReadDir
method to get a listing of the files. The listing also includes directories, so we will use the IsDir
method to determine whether we should skip it or not.
We will then check to see whether each file starts with the userid
field (remember that we named our files in this way) by a call to path.Match
. If the filename matches the userid
field, then we have found the file for that user and we return the path. If anything goes wrong or if we can't find the file, we return the ErrNoAvatarURL
error as usual.
Update the appropriate method in avatar.go
with the following code:
func (_ FileSystemAvatar) GetAvatarURL(c *client) (string, error) { if userid, ok := c.userData["userid"]; ok { if useridStr, ok := userid.(string); ok { if files, err := ioutil.ReadDir("avatars"); err == nil { for _, file := range files { if file.IsDir() { continue } if match, _ := path.Match(useridStr+"*", file.Name()); match { return "/avatars/" + file.Name(), nil } } } } } return "", ErrNoAvatarURL }
Delete all the files in the avatar
folder to prevent confusion and rebuild the program. This time upload an image of a different type and notice that our application has no difficulty handling it.
When we look back at how our Avatar
type is used, you will notice that every time someone sends a message, the application makes a call to GetAvatarURL
. In our latest implementation, each time the method is called, we iterate over all the files in the avatars
folder. For a particularly chatty user, this could mean that we end up iterating over and over again many times a minute. This is an obvious waste of resources and would, at some point very soon, become a scaling problem.
Instead of getting the avatar URL for every message, we will get it only once when the user first logs in and cache it in the auth
cookie. Unfortunately, our Avatar
interface type requires that we pass in a client
object to the GetAvatarURL
method and we do not have such an object at the point at which we are authenticating the user.
So did we make a mistake when we designed our Avatar
interface? While this is a natural conclusion to come to, in fact we did the right thing. We designed the solution with the best information we had available at the time and therefore had a working chat application much sooner than if we'd tried to design for every possible future case. Software evolves and almost always changes during the development process and will definitely change throughout the lifetime of the code.
We have concluded that our GetAvatarURL
method depends on a type that is not available to us at the point we need it, so what would be a good alternative? We could pass each required field as a separate argument but this would make our interface brittle, since as soon as an Avatar
implementation needs a new piece of information, we'd have to change the method signature. Instead, we will create a new type that will encapsulate the information our Avatar
implementations need while conceptually remaining decoupled from our specific case.
In auth.go
, add the following code to the top of the page (underneath the package
keyword of course):
import gomniauthcommon "github.com/stretchr/gomniauth/common" type ChatUser interface { UniqueID() string AvatarURL() string } type chatUser struct { gomniauthcommon.User uniqueID string } func (u chatUser) UniqueID() string { return u.uniqueID }
Here, the import
statement imported the common
package from Gomniauth and at the same time gave it a specific name through which it will be accessed: gomniauthcommon
. This isn't entirely necessary since we have no package name conflicts. However, it makes the code easier to understand.
In the preceding code snippet, we also defined a new interface type called ChatUser
, which exposes the information needed in order for our Avatar
implementations to generate the correct URLs. Then, we defined an actual implementation called chatUser
(notice the lowercase starting letter) that implements the interface. It also makes use of a very interesting feature in Go: type embedding. We actually embedded the interface type gomniauth/common.User
, which means that our struct
implements the interface automatically.
You may have noticed that we only actually implemented one of the two required methods to satisfy our ChatUser
interface. We got away with this because the Gomniauth User
interface happens to define the same AvatarURL
method. In practice, when we instantiate our chatUser
struct—provided we set an appropriate value for the implied Gomniauth User
field—our object implements both Gomniauth's User
interface and our own ChatUser
interface at the same time.
Before we can use our new type, we must update the Avatar
interface and appropriate implementations to make use of it. As we will follow TDD practices, we are going to make these changes in our test file, see compiler errors when we try to build our code, and see failing tests once we fix those errors before finally making the tests pass.
Open avatar_test.go
and replace TestAuthAvatar
with the following code:
func TestAuthAvatar(t *testing.T) { var authAvatar AuthAvatar testUser := &gomniauthtest.TestUser{} testUser.On("AvatarURL").Return("", ErrNoAvatarURL) testChatUser := &chatUser{User: testUser} url, err := authAvatar.GetAvatarURL(testChatUser) if err != ErrNoAvatarURL { t.Error("AuthAvatar.GetAvatarURL should return ErrNoAvatarURL when no value present") } testUrl := "http://url-to-gravatar/" testUser = &gomniauthtest.TestUser{} testChatUser.User = testUser testUser.On("AvatarURL").Return(testUrl, nil) url, err = authAvatar.GetAvatarURL(testChatUser) if err != nil { t.Error("AuthAvatar.GetAvatarURL should return no error when value present") } else { if url != testUrl { t.Error("AuthAvatar.GetAvatarURL should return correct URL") } } }
Using our new interface before we have defined it is a good way to check the sanity of our thinking, which is another advantage of practicing TDD. In this new test, we create TestUser
provided by Gomniauth and embed it into a chatUser
type. We then pass the new chatUser
type into our GetAvatarURL
calls and make the same assertions about output as we always have done.
Gomniauth's TestUser
type is interesting as it makes use of the Testify
package's mocking capabilities. See https://github.com/stretchr/testify for more information.
The On
and Return
methods allow us to tell TestUser
what to do when specific methods are called. In the first case, we tell the AvatarURL
method to return the error, and in the second case, we ask it to return the testUrl
value, which simulates the two possible outcomes we are covering in this test.
Updating the TestGravatarAvatar
and TestFileSystemAvatar
tests is much simpler because they rely only on the UniqueID
method, the value of which we can control directly.
Replace the other two tests in avatar_test.go
with the following code:
func TestGravatarAvatar(t *testing.T) { var gravatarAvitar GravatarAvatar user := &chatUser{uniqueID: "abc"} url, err := gravatarAvitar.GetAvatarURL(user) if err != nil { t.Error("GravatarAvitar.GetAvatarURL should not return an error") } if url != "//www.gravatar.com/avatar/abc" { t.Errorf("GravatarAvitar.GetAvatarURL wrongly returned %s", url) } } func TestFileSystemAvatar(t *testing.T) { // make a test avatar file filename := path.Join("avatars", "abc.jpg") ioutil.WriteFile(filename, []byte{}, 0777) defer func() { os.Remove(filename) }() var fileSystemAvatar FileSystemAvatar user := &chatUser{uniqueID: "abc"} url, err := fileSystemAvatar.GetAvatarURL(user) if err != nil { t.Error("FileSystemAvatar.GetAvatarURL should not return an error") } if url != "/avatars/abc.jpg" { t.Errorf("FileSystemAvatar.GetAvatarURL wrongly returned %s", url) } }
Of course, this test code won't even compile because we are yet to update our Avatar
interface. In avatar.go
, update the GetAvatarURL
signature in the Avatar
interface type to take a ChatUser
type rather than a client
type:
GetAvatarURL(ChatUser) (string, error)
Trying to build this will reveal that we now have broken implementations because all the GetAvatarURL
methods are still asking for a client
object.
Changing an interface like the one we have is a good way to automatically find the parts of our code that have been affected because they will cause compiler errors. Of course, if we were writing a package that other people would use, we would have to be far stricter towards changing the interfaces.
We are now going to update the three implementation signatures to satisfy the new interface and change the method bodies to make use of the new type. Replace the implementation for FileSystemAvatar
with the following:
func (_ FileSystemAvatar) GetAvatarURL(u ChatUser) (string, error) { if files, err := ioutil.ReadDir("avatars"); err == nil { for _, file := range files { if file.IsDir() { continue } if match, _ := path.Match(u.UniqueID()+"*", file.Name()); match { return "/avatars/" + file.Name(), nil } } } return "", ErrNoAvatarURL }
The key change here is that we no longer access the userData
field on the client, and instead just call UniqueID
directly on the ChatUser
interface.
Next, we update the AuthAvatar
implementation with the following code:
func (_ AuthAvatar) GetAvatarURL(u ChatUser) (string, error) { url := u.AvatarURL() if len(url) > 0 { return url, nil } return "", ErrNoAvatarURL }
Our new design is proving to be much simpler; it's always a good thing if we can reduce the amount of code needed. The preceding code makes a call to get the AvatarURL
value, and provided it isn't empty (or len(url) > 0
), we return it; else, we return the ErrNoAvatarURL
error instead.
Finally, update the GravatarAvatar
implementation:
func (_ GravatarAvatar) GetAvatarURL(u ChatUser) (string, error) { return "//www.gravatar.com/avatar/" + u.UniqueID(), nil }
So far, we have assigned the Avatar
implementation to the room
type, which enables us to use different avatars for different rooms. However, this has exposed an issue: when our users sign in, there is no concept of which room they are headed to so we cannot know which Avatar
implementation to use. Because our application only supports a single room, we are going to look at another approach toward selecting implementations: the use of global variables.
A global variable is simply a variable that is defined outside any type definition and is accessible from every part of the package (and from outside the package if it's exported). For a simple configuration, such as which type of Avatar
implementation to use, they are an easy and simple solution. Underneath the import
statements in main.go
, add the following line:
// set the active Avatar implementation var avatars Avatar = UseFileSystemAvatar
This defines avatars
as a global variable that we can use when we need to get the avatar URL for a particular user.
We need to change the code that calls GetAvatarURL
for every message to just access the value that we put into the userData
cache (via the auth
cookie). Change the line where msg.AvatarURL
is assigned, as follows:
if avatarUrl, ok := c.userData["avatar_url"]; ok { msg.AvatarURL = avatarUrl.(string) }
Find the code inside loginHandler
in auth.go
where we call provider.GetUser
and replace it down to where we set the authCookieValue
object with the following code:
user, err := provider.GetUser(creds) if err != nil { log.Fatalln("Error when trying to get user from", provider, "-", err) } chatUser := &chatUser{User: user} m := md5.New() io.WriteString(m, strings.ToLower(user.Name())) chatUser.uniqueID = fmt.Sprintf("%x", m.Sum(nil)) avatarURL, err := avatars.GetAvatarURL(chatUser) if err != nil { log.Fatalln("Error when trying to GetAvatarURL", "-", err) }
Here, we created a new chatUser
variable while setting the User
field (which represents the embedded interface) to the User
value returned from Gomniauth. We then saved the userid
MD5 hash to the uniqueID
field.
The call to avatars.GetAvatarURL
is where all of our hard work has paid off, as we now get the avatar URL for the user far earlier in the process. Update the authCookieValue
line in auth.go
to cache the avatar URL in the cookie and remove the e-mail address since it is no longer needed:
authCookieValue := objx.New(map[string]interface{}{ "userid": chatUser.uniqueID, "name": user.Name(), "avatar_url": avatarURL, }).MustBase64()
However expensive the work that the Avatar
implementation needs to do, like iterating over files on the filesystem, it is mitigated by the fact that the implementation only does so when the user first logs in, and not every time they send a message.
Finally, we get to snip away some of the fat that has accumulated during our refactoring process.
Since we no longer store the Avatar
implementation in room
, let's remove the field and all references to it from the type. In room.go
, delete the avatar Avatar
definition from the room
struct and update the newRoom
method:
func newRoom() *room { return &room{ forward: make(chan *message), join: make(chan *client), leave: make(chan *client), clients: make(map[*client]bool), tracer: trace.Off(), } }
In main.go
, remove the parameter passed into the newRoom
function call since we are using our global variable instead of this one.
After this exercise, the end user experience remains unchanged. Usually, when refactoring the code, it is the internals that are modified while the public-facing interface remains stable and unchanged.