Go Memory FSs Everywhere in Test: Treat Side Effects as Dependencies
If your Go code calls os.Create, os.Stat, exec.LookPath, or net.Dial directly from business logic, your tests are forced to touch real side effects more often than necessary.
The pattern in this article is simple:
- Define small interfaces for side effects.
- Inject real implementations in production.
- Inject in-memory fakes or standard library memory file systems in tests.
This is why I encourage myfs.Stat("file.txt") (or whatever your project names it) over direct os.Stat("file.txt") in your core application code. In-memory file systems provide deterministic, fast execution for testing.
Why memory file systems matter for switching implementations
The key benefit is switchability without refactoring call sites.
If your code is hard-coded to os.*, switching behavior later is expensive. If your code depends on an interface, switching is just wiring. Using an in-memory file system like fstest.MapFS makes testing simpler.
- production: use real OS-backed implementation,
- unit tests: use in-memory fake,
- integration tests: use real disk with temp dirs,
- special tools: use embedded FS or
fstest.MapFS.
This gives you speed today and flexibility later.
Why most (but not all) tests should use memory file systems
You usually want:
- many unit tests with no real FS/network/process calls by using in-memory implementations,
- fewer integration tests that validate real FS and OS behavior,
- a small number of end-to-end tests for full confidence.
This split gives better feedback loops:
- fast local iteration,
- fewer flaky tests,
- better coverage of failure branches,
- still keeping real-world verification where it matters.
So this is not “never touch the file system.” It is “use real side effects intentionally, and memory file systems by default.”
Core pattern: wrap, inject, test with mockFS
1) Define the minimal interface your code needs
1package myfs
2
3import (
4 "io"
5 "io/fs"
6 "os"
7)
8
9// FS only includes operations this package needs.
10type FS interface {
11 Create(name string) (io.WriteCloser, error)
12 Stat(name string) (fs.FileInfo, error)
13 ReadFile(name string) ([]byte, error)
14}
15
16// OSFS is the production implementation.
17type OSFS struct{}
18
19func (OSFS) Create(name string) (io.WriteCloser, error) { return os.Create(name) }
20func (OSFS) Stat(name string) (fs.FileInfo, error) { return os.Stat(name) }
21func (OSFS) ReadFile(name string) ([]byte, error) { return os.ReadFile(name) }
2) Inject the interface into business logic
1package report
2
3import (
4 "errors"
5 "fmt"
6 "path/filepath"
7
8 "example.com/app/myfs"
9)
10
11type Generator struct {
12 FS myfs.FS
13}
14
15func NewGenerator(fs myfs.FS) *Generator {
16 if fs == nil {
17 fs = myfs.OSFS{}
18 }
19 return &Generator{FS: fs}
20}
21
22func (g *Generator) Write(output string, data []byte) error {
23 if filepath.Base(output) != output {
24 return errors.New("security error: output file must be in the current directory")
25 }
26
27 f, err := g.FS.Create(output)
28 if err != nil {
29 return fmt.Errorf("create output: %w", err)
30 }
31 defer f.Close()
32
33 if _, err := f.Write(data); err != nil {
34 return fmt.Errorf("write output: %w", err)
35 }
36 return nil
37}
3) Test with in-memory fakes in idiomatic table-driven tests
1package report_test
2
3import (
4 "bytes"
5 "errors"
6 "io"
7 "io/fs"
8 "strings"
9 "testing"
10 "time"
11
12 "example.com/app/report"
13)
14
15type nopCloser struct{ io.Writer }
16
17func (nopCloser) Close() error { return nil }
18
19type fakeInfo struct{ name string }
20
21func (f fakeInfo) Name() string { return f.name }
22func (f fakeInfo) Size() int64 { return 0 }
23func (f fakeInfo) Mode() fs.FileMode { return 0 }
24func (f fakeInfo) ModTime() time.Time { return time.Time{} }
25func (f fakeInfo) IsDir() bool { return false }
26func (f fakeInfo) Sys() any { return nil }
27
28type mockFS struct {
29 files map[string]*bytes.Buffer
30 createErr error
31}
32
33func (m *mockFS) Create(name string) (io.WriteCloser, error) {
34 if m.createErr != nil {
35 return nil, m.createErr
36 }
37 if m.files == nil {
38 m.files = map[string]*bytes.Buffer{}
39 }
40 buf := new(bytes.Buffer)
41 m.files[name] = buf
42 return nopCloser{Writer: buf}, nil
43}
44
45func (m *mockFS) Stat(name string) (fs.FileInfo, error) {
46 if _, ok := m.files[name]; !ok {
47 return nil, fs.ErrNotExist
48 }
49 return fakeInfo{name: name}, nil
50}
51
52func (m *mockFS) ReadFile(name string) ([]byte, error) {
53 f, ok := m.files[name]
54 if !ok {
55 return nil, fs.ErrNotExist
56 }
57 return f.Bytes(), nil
58}
59
60func TestGeneratorWrite(t *testing.T) {
61 tests := []struct {
62 name string
63 path string
64 data []byte
65 fs *mockFS
66 wantErr string
67 }{
68 {
69 name: "reject parent path",
70 path: "../report.md",
71 data: []byte("x"),
72 fs: &mockFS{},
73 wantErr: "security error",
74 },
75 {
76 name: "reject absolute path",
77 path: "/tmp/report.md",
78 data: []byte("x"),
79 fs: &mockFS{},
80 wantErr: "security error",
81 },
82 {
83 name: "create failure",
84 path: "report.md",
85 data: []byte("x"),
86 fs: &mockFS{createErr: errors.New("permission denied")},
87 wantErr: "create output",
88 },
89 {
90 name: "success",
91 path: "report.md",
92 data: []byte("hello"),
93 fs: &mockFS{},
94 },
95 }
96
97 for _, tc := range tests {
98 t.Run(tc.name, func(t *testing.T) {
99 g := report.NewGenerator(tc.fs)
100 err := g.Write(tc.path, tc.data)
101 if tc.wantErr != "" {
102 if err == nil || !strings.Contains(err.Error(), tc.wantErr) {
103 t.Fatalf("expected error containing %q, got %v", tc.wantErr, err)
104 }
105 return
106 }
107 if err != nil {
108 t.Fatalf("unexpected error: %v", err)
109 }
110 if got := tc.fs.files[tc.path].String(); got != string(tc.data) {
111 t.Fatalf("wrote %q, want %q", got, string(tc.data))
112 }
113 })
114 }
115}
Why myfs.Stat over direct os.Stat in app code
Using your own interface boundary means your package owns behavior and can evolve safely.
With a project-local wrapper (myfs, deps, platform, etc.), you can:
- enforce security or path policy centrally,
- add metrics/logging in one place,
- simulate errors in tests,
- migrate storage backends later,
- keep unit tests fast by default using memory.
Direct os.Stat spread across the codebase makes all of that harder.
Standard library memory-friendly FS tools (larger examples)
Example A: fstest.MapFS for read-only parser tests
1package parser_test
2
3import (
4 "io/fs"
5 "testing"
6 "testing/fstest"
7)
8
9func countGoFiles(fsys fs.FS) (int, error) {
10 n := 0
11 err := fs.WalkDir(fsys, ".", func(path string, d fs.DirEntry, err error) error {
12 if err != nil {
13 return err
14 }
15 if !d.IsDir() && len(path) > 3 && path[len(path)-3:] == ".go" {
16 n++
17 }
18 return nil
19 })
20 return n, err
21}
22
23func TestCountGoFiles_MapFS(t *testing.T) {
24 fsys := fstest.MapFS{
25 "go.mod": &fstest.MapFile{Data: []byte("module example.com/test\n")},
26 "main.go": &fstest.MapFile{Data: []byte("package main\n")},
27 "pkg/a.go": &fstest.MapFile{Data: []byte("package pkg\n")},
28 "pkg/README.md": &fstest.MapFile{Data: []byte("docs\n")},
29 }
30
31 got, err := countGoFiles(fsys)
32 if err != nil {
33 t.Fatalf("countGoFiles failed: %v", err)
34 }
35 if got != 2 {
36 t.Fatalf("got %d go files, want 2", got)
37 }
38}
Example B: table-driven path filtering on fstest.MapFS
1func findMatching(fsys fs.FS, suffix string) ([]string, error) {
2 var out []string
3 err := fs.WalkDir(fsys, ".", func(path string, d fs.DirEntry, err error) error {
4 if err != nil {
5 return err
6 }
7 if d.IsDir() {
8 return nil
9 }
10 if strings.HasSuffix(path, suffix) {
11 out = append(out, path)
12 }
13 return nil
14 })
15 return out, err
16}
17
18func TestFindMatching_MapFS(t *testing.T) {
19 base := fstest.MapFS{
20 "pkg/a.go": &fstest.MapFile{Data: []byte("a")},
21 "pkg/b_test.go": &fstest.MapFile{Data: []byte("b")},
22 "notes.txt": &fstest.MapFile{Data: []byte("x")},
23 }
24
25 tests := []struct {
26 name string
27 suffix string
28 want int
29 }{
30 {name: "go", suffix: ".go", want: 2},
31 {name: "test", suffix: "_test.go", want: 1},
32 {name: "none", suffix: ".md", want: 0},
33 }
34
35 for _, tc := range tests {
36 t.Run(tc.name, func(t *testing.T) {
37 got, err := findMatching(base, tc.suffix)
38 if err != nil {
39 t.Fatalf("findMatching failed: %v", err)
40 }
41 if len(got) != tc.want {
42 t.Fatalf("len(got)=%d want=%d (%v)", len(got), tc.want, got)
43 }
44 })
45 }
46}
Example C: real FS integration test with t.TempDir
1func TestCountGoFiles_Integration(t *testing.T) {
2 dir := t.TempDir()
3 if err := os.WriteFile(filepath.Join(dir, "main.go"), []byte("package main\n"), 0o644); err != nil {
4 t.Fatalf("write main.go: %v", err)
5 }
6 if err := os.WriteFile(filepath.Join(dir, "README.md"), []byte("docs\n"), 0o644); err != nil {
7 t.Fatalf("write README.md: %v", err)
8 }
9
10 got, err := countGoFiles(os.DirFS(dir))
11 if err != nil {
12 t.Fatalf("countGoFiles failed: %v", err)
13 }
14 if got != 1 {
15 t.Fatalf("got %d, want 1", got)
16 }
17}
Use MapFS for most tests, then keep a smaller integration layer like this.
Same design for exec, sockets, and HTTP
This pattern is not file-system specific.
Cleaner execProxy with embedded Discoverer
1type execProxy interface {
2 Command(name string, arg ...string) *exec.Cmd
3 LookPath(file string) (string, error)
4}
5
6type Discoverer struct {
7 ExecCommand func(name string, arg ...string) *exec.Cmd
8 ExecLookPath func(file string) (string, error)
9}
10
11type realExecProxy struct{ Discoverer }
12
13func newRealExecProxy() realExecProxy {
14 return realExecProxy{Discoverer: Discoverer{
15 ExecCommand: exec.Command,
16 ExecLookPath: exec.LookPath,
17 }}
18}
19
20func (r realExecProxy) Command(name string, arg ...string) *exec.Cmd {
21 return r.ExecCommand(name, arg...)
22}
23
24func (r realExecProxy) LookPath(file string) (string, error) {
25 return r.ExecLookPath(file)
26}
This keeps injection explicit without global mutable variables.
Table-driven test for exec discovery
1type mockExecProxy struct {
2 commandFn func(name string, arg ...string) *exec.Cmd
3 lookPathFn func(file string) (string, error)
4}
5
6func (m mockExecProxy) Command(name string, arg ...string) *exec.Cmd {
7 return m.commandFn(name, arg...)
8}
9
10func (m mockExecProxy) LookPath(file string) (string, error) {
11 return m.lookPathFn(file)
12}
13
14func TestDiscover_PathLookup(t *testing.T) {
15 tests := []struct {
16 name string
17 proxy execProxy
18 wantErr bool
19 }{
20 {
21 name: "found",
22 proxy: mockExecProxy{lookPathFn: func(file string) (string, error) {
23 return "/mock/" + file, nil
24 }},
25 },
26 {
27 name: "missing",
28 proxy: mockExecProxy{lookPathFn: func(file string) (string, error) {
29 return "", exec.ErrNotFound
30 }},
31 wantErr: true,
32 },
33 }
34
35 for _, tc := range tests {
36 t.Run(tc.name, func(t *testing.T) {
37 _, err := tc.proxy.LookPath("git")
38 if tc.wantErr && err == nil {
39 t.Fatal("expected error")
40 }
41 if !tc.wantErr && err != nil {
42 t.Fatalf("unexpected error: %v", err)
43 }
44 })
45 }
46}
Sockets and HTTP
Use the same idea for networking:
- inject a
Dialerinterface instead of hard-codingnet.Dialer, - use
net.Pipe()or fake dialers in unit tests, - use
httptest.NewServerfor controlled HTTP integration tests.
Many teams already do this with httptest; this article is about applying that same discipline consistently across all side effects.
Copy-paste starters
Starter 1: minimal FS wrapper + constructor injection
1package deps
2
3import (
4 "io"
5 "io/fs"
6 "os"
7)
8
9type FileSystem interface {
10 Create(name string) (io.WriteCloser, error)
11 Stat(name string) (fs.FileInfo, error)
12}
13
14type OSFS struct{}
15
16func (OSFS) Create(name string) (io.WriteCloser, error) { return os.Create(name) }
17func (OSFS) Stat(name string) (fs.FileInfo, error) { return os.Stat(name) }
1package app
2
3type Service struct {
4 FS deps.FileSystem
5}
6
7func NewService(fs deps.FileSystem) *Service {
8 if fs == nil {
9 fs = deps.OSFS{}
10 }
11 return &Service{FS: fs}
12}
Starter 2: in-memory fake + table-driven test skeleton
1type nopCloser struct{ io.Writer }
2
3func (nopCloser) Close() error { return nil }
4
5type fakeFS struct {
6 createErr error
7 files map[string]*bytes.Buffer
8}
9
10func (f *fakeFS) Create(name string) (io.WriteCloser, error) {
11 if f.createErr != nil {
12 return nil, f.createErr
13 }
14 if f.files == nil {
15 f.files = map[string]*bytes.Buffer{}
16 }
17 buf := new(bytes.Buffer)
18 f.files[name] = buf
19 return nopCloser{Writer: buf}, nil
20}
21
22func TestService(t *testing.T) {
23 tests := []struct {
24 name string
25 fs *fakeFS
26 wantErr bool
27 }{
28 {name: "ok", fs: &fakeFS{}},
29 {name: "create fails", fs: &fakeFS{createErr: errors.New("boom")}, wantErr: true},
30 }
31
32 for _, tc := range tests {
33 t.Run(tc.name, func(t *testing.T) {
34 svc := NewService(tc.fs)
35 err := svc.DoWork()
36 if tc.wantErr && err == nil {
37 t.Fatal("expected error")
38 }
39 if !tc.wantErr && err != nil {
40 t.Fatalf("unexpected error: %v", err)
41 }
42 })
43 }
44}
Starter 3: exec wrapper with explicit function injection
1type Exec interface {
2 Command(name string, arg ...string) *exec.Cmd
3 LookPath(file string) (string, error)
4}
5
6type RealExec struct {
7 CommandFn func(name string, arg ...string) *exec.Cmd
8 LookPathFn func(file string) (string, error)
9}
10
11func NewRealExec() RealExec {
12 return RealExec{CommandFn: exec.Command, LookPathFn: exec.LookPath}
13}
14
15func (r RealExec) Command(name string, arg ...string) *exec.Cmd { return r.CommandFn(name, arg...) }
16func (r RealExec) LookPath(file string) (string, error) { return r.LookPathFn(file) }
Final guideline
If code touches the outside world, make that capability an interface dependency.
- production wires real implementations,
- most tests wire fake/in-memory implementations,
- a smaller integration suite verifies real behavior.
That balance gives fast tests, cleaner architecture, and safer refactoring.