27 Commits

Author SHA1 Message Date
Yoshiyuki Mineo
2343cac676 Check compose args (#74)
* v2: take only 'bitsMachine' least significant bits from generated machine IDs so it doesn't corrupt the other ID parts (#72)

Co-authored-by: ok32 <e.starikoff@gmail.com>
Co-authored-by: Yoshiyuki Mineo <Yoshiyuki.Mineo@jp.sony.com>

* Refactor Sonyflake Compose method to return errors for invalid parameters and update related tests. Consolidate error handling for start time, sequence, and machine ID validations. Remove unused variable in tests for clarity.

---------

Co-authored-by: ok32 <artuh.gubanova@gmail.com>
Co-authored-by: ok32 <e.starikoff@gmail.com>
2025-05-18 15:56:36 +09:00
Yoshiyuki Mineo
5347433c8c Enhance Sonyflake error handling by adding tests for invalid machine IDs, including cases for too large and negative values. (#73) 2025-05-18 15:16:04 +09:00
Yoshiyuki Mineo
774342570a Add Compose method (#71) 2025-05-07 13:17:30 +09:00
Yoshiyuki Mineo
59c47aeab1 Fix lint errors (#70) 2025-05-05 19:51:28 +09:00
Yoshiyuki Mineo
d764be18d5 Update README.md (#69)
* Update README.md

* Update README.md

* Update test-v2.yml

* Update test-v1.yml
2025-05-05 14:46:40 +09:00
Yoshiyuki Mineo
7ee8f154df feat(v2): make bit assignment for time/sequence/machine customizable … (#68)
* feat(v2): make bit assignment for time/sequence/machine customizable via Settings; update all logic and tests

* gofmt
2025-05-05 14:16:21 +09:00
Yoshiyuki Mineo
357b2ee5f0 Upgrade Sonyflake to v2 (#66)
* Add v2

* Introduce staticcheck

* Introduce golangci-lint

* Add error checks

* Add error checks

* Lint v2 code

* Improve CI trigger

* Use io.ReadAll

* Use int64

* Remove NewSonyflake

* Fix errors

* v2: Change MachineID, sequence, and AmazonEC2MachineID to int; update all usage and tests for type consistency

* docs: update Settings struct in README to use int for MachineID and CheckMachineID (v2)

* docs(v2): clarify Settings, StartTime, MachineID, and CheckMachineID comments and update README links and explanations

* docs(v2/mock): improve comments and docstrings for mock implementations

* docs(types): unify and clarify package and type docstrings for types.go in v1 and v2

* test(v2): refactor and modernize tests, improve error assertions, and update mocks for v2

* test(v2): normalize whitespace in pseudoSleep calls for consistency

* feat(v2): add configurable TimeUnit and refactor time handling for So… (#67)

* feat(v2): add configurable TimeUnit and refactor time handling for Sonyflake v2

* test(v2): add ToTime tests, clarify TimeUnit behavior, and update docs for v2

* gofmt
2025-05-05 10:53:08 +09:00
Yoshiyuki Mineo
8d195df6f7 Fix a missing link (#65) 2025-05-01 19:25:49 +09:00
Yoshiyuki Mineo
410eb250e3 Update Go versions (#64) 2025-05-01 19:15:37 +09:00
Yoshiyuki Mineo
94f43cfd99 Update Go versions (#60) 2024-10-13 22:30:52 +09:00
Yoshiyuki Mineo
b9b40b47a5 Update go versions (#54) 2024-04-30 15:50:36 +09:00
Yoshiyuki Mineo
a0558cef64 Link local (#53)
* Allow IPv4 Link Local addresses (#50)

* Allow IPv4 Link Local addresses

Allow the use of link local addresses

* Update sonyflake.go

* Update sonyflake.go

* Update a comment

---------

Co-authored-by: Flavio Crisciani <f.crisciani@gmail.com>
2024-04-30 15:06:58 +09:00
David E. Wheeler
fc2f84a086 Use net.IP.Equal instead of bytes.Equal (#49)
As suggested by go-staticcheck.
2023-11-06 14:32:46 +09:00
Yoshiyuki Mineo
06f9b47996 Introduce New function (#47)
* feat(Sonyflake): define error variables

* feat(Sonyflake): add New() function
- minor logic improvements
- return errors

* tests(Sonyflake): remove old TestNilSonyflake test function in favour of the New() function coverage

* gofmt

* Update error messages and comments

* Introduce New function

---------

Co-authored-by: Quetzy Garcia <quetzy.garcia@integrate.com>
2023-08-14 01:27:55 +09:00
Yoshiyuki Mineo
18c4908321 Update go versions 2023-08-13 23:36:13 +09:00
Yoshiyuki Mineo
eafab81cd5 Update go versions (#43) 2023-05-04 02:08:54 +09:00
Yoshiyuki Mineo
597171da2e Increase coverage (#42)
* Introduce mocking framework and increase coverage (#22)

This change introduces two common golang patterns:

- types: this will allow fine-tuned control over imported
types by defining where they will be used and how

- mock: this allows the generation of mock constructors,
which allows for testing any individual path in a method by
"injecting" a mock method which matches the expected type

This change also increases test coverage to 100%

Co-authored-by: Yoshiyuki Mineo <Yoshiyuki.Mineo@jp.sony.com>

* gofmt

---------

Co-authored-by: Bradley Boutcher <btboutcher@icloud.com>
2023-05-04 00:08:20 +09:00
Yoshiyuki Mineo
cc94b60628 Check time order explicitly (#37)
* Add sortable test (#11)

* add TestSortableID to make sure that generated ID(s) can be sorted like you are using increment id database

* add TestSortableID to make sure that generated ID(s) can be sorted like you are using increment id database

Co-authored-by: Yoshiyuki Mineo <Yoshiyuki.Mineo@jp.sony.com>

* gofmt

* Check time order explicitly

Co-authored-by: Yusuf Syaifudin <yusuf.syaifudin@gmail.com>
2022-08-17 14:33:52 +09:00
Yoshiyuki Mineo
3719d006ac Add a note (#36) 2022-08-13 00:23:02 +09:00
Yoshiyuki Mineo
90e18212ad Add functions to get ID elements (#35)
* Refactoring

* Add functions to get ID elements
2022-08-12 18:42:10 +09:00
Yoshiyuki Mineo
809c515cc5 Introduce GitHub Actions (#34)
* Introduce GitHub Actions

* gofmt
2022-08-12 18:38:40 +09:00
Osamu TONOMORI
848d664cee Make testing simple (#20)
* Make testing simpler

* Remove testing deps

* Simplemize tests
2020-08-27 10:17:19 +09:00
Yoshiyuki Mineo
60e9d38e92 Merge pull request #19 from osamingo/set-go-stable-versions
Set Go stable versions to .travis.yml
2020-08-18 16:53:08 +09:00
Osamu TONOMORI
f68244fede Set Go stable versions 2020-08-10 16:05:22 +09:00
Osamu TONOMORI
963f058659 Remove sudo setting
Setting `sudo: false` is not recommended.
via/
https://blog.travis-ci.com/2018-11-19-required-linux-infrastructure-migration
2020-08-10 16:03:44 +09:00
Yoshiyuki Mineo
59cd942daa Update README.md 2019-10-07 16:50:43 +09:00
Yoshiyuki Mineo
3ffd8c4254 Update README.md 2019-10-07 16:40:37 +09:00
25 changed files with 1477 additions and 140 deletions

49
.github/workflows/test-v1.yml vendored Normal file
View File

@@ -0,0 +1,49 @@
name: CI for v1 Module
on:
push:
paths-ignore:
- README.md
- .github/workflows/test-v2.yml
- v2/**
pull_request:
paths-ignore:
- README.md
- .github/workflows/test-v2.yml
- v2/**
permissions: read-all
jobs:
test-v1:
strategy:
matrix:
go-version: [1.23.x, 1.24.x]
os: [ubuntu-latest]
runs-on: ${{matrix.os}}
defaults:
run:
working-directory: ./
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version: ${{matrix.go-version}}
- name: Lint Code
uses: golangci/golangci-lint-action@v6
- name: Check format
run: test -z "`gofmt -l .`"
- name: Run tests
run: go test -v ./...
- name: Build example
run: cd example && ./linux64_build.sh

54
.github/workflows/test-v2.yml vendored Normal file
View File

@@ -0,0 +1,54 @@
name: CI for v2 Module
on:
push:
paths:
- README.md
- .github/workflows/test-v2.yml
- v2/**
pull_request:
paths:
- README.md
- .github/workflows/test-v2.yml
- v2/**
permissions: read-all
jobs:
test-v2:
strategy:
matrix:
go-version: [1.23.x, 1.24.x]
os: [ubuntu-latest]
runs-on: ${{matrix.os}}
defaults:
run:
working-directory: ./v2
steps:
- name: Checkout code
uses: actions/checkout@v4
- name: Set up Go
uses: actions/setup-go@v5
with:
go-version-file: ./v2/go.mod
- name: Tidy modules
run: go mod tidy
- name: Lint Code
uses: golangci/golangci-lint-action@v6
with:
working-directory: ./v2
- name: Check format
run: test -z "`gofmt -l .`"
- name: Run tests
run: go test ./... -v
- name: Build example
run: cd example && ./linux64_build.sh

3
.gitignore vendored
View File

@@ -1 +1,4 @@
example/example
example/sonyflake_server
v2/example/example
v2/example/sonyflake_server

View File

@@ -1,14 +0,0 @@
language: go
go:
- 1.10.x
- 1.11.x
sudo: false
before_install:
- go get -u golang.org/x/lint/golint
- go get github.com/axw/gocov/gocov
- go get github.com/mattn/goveralls
script:
- test -z "`gofmt -l .`"
- test -z "`golint ./...`"
- $GOPATH/bin/goveralls -service=travis-ci
- cd example && ./linux64_build.sh

View File

@@ -1,84 +1,109 @@
Sonyflake
=========
# Sonyflake
[![GoDoc](https://godoc.org/github.com/sony/sonyflake?status.svg)](http://godoc.org/github.com/sony/sonyflake)
[![Build Status](https://travis-ci.org/sony/sonyflake.svg?branch=master)](https://travis-ci.org/sony/sonyflake)
[![Coverage Status](https://coveralls.io/repos/sony/sonyflake/badge.svg?branch=master&service=github)](https://coveralls.io/github/sony/sonyflake?branch=master)
[![Go Report Card](https://goreportcard.com/badge/github.com/sony/sonyflake)](https://goreportcard.com/report/github.com/sony/sonyflake)
[![GoDoc](https://pkg.go.dev/badge/github.com/sony/sonyflake/v2?utm_source=godoc)](https://pkg.go.dev/github.com/sony/sonyflake/v2)
[![Go Report Card](https://goreportcard.com/badge/github.com/sony/sonyflake/v2)](https://goreportcard.com/report/github.com/sony/sonyflake/v2)
Sonyflake is a distributed unique ID generator inspired by [Twitter's Snowflake](https://blog.twitter.com/2010/announcing-snowflake).
A Sonyflake ID is composed of
Sonyflake focuses on lifetime and performance on many host/core environment.
So it has a different bit assignment from Snowflake.
By default, a Sonyflake ID is composed of
39 bits for time in units of 10 msec
8 bits for a sequence number
16 bits for a machine id
Installation
------------
As a result, Sonyflake has the following advantages and disadvantages:
- The lifetime (174 years) is longer than that of Snowflake (69 years)
- It can work in more distributed machines (2^16) than Snowflake (2^10)
- It can generate 2^8 IDs per 10 msec at most in a single instance (fewer than Snowflake)
However, if you want more generation rate in a single host,
you can easily run multiple Sonyflake instances parallelly using goroutines.
In addition, you can adjust the lifetime and generation rate of Sonyflake
by customizing the bit assignment and the time unit.
## Installation
```
go get github.com/sony/sonyflake
go get github.com/sony/sonyflake/v2
```
Usage
-----
## Usage
The function NewSonyflake creates a new Sonyflake instance.
The function New creates a new Sonyflake instance.
```go
func NewSonyflake(st Settings) *Sonyflake
func New(st Settings) (*Sonyflake, error)
```
You can configure Sonyflake by the struct Settings:
```go
type Settings struct {
BitsSequence int
BitsMachineID int
TimeUnit time.Duration
StartTime time.Time
MachineID func() (uint16, error)
CheckMachineID func(uint16) bool
MachineID func() (int, error)
CheckMachineID func(int) bool
}
```
- BitsSequence is the bit length of a sequence number.
If BitsSequence is 0, the default bit length is used, which is 8.
If BitsSequence is 31 or more, an error is returned.
- BitsMachineID is the bit length of a machine ID.
If BitsMachineID is 0, the default bit length is used, which is 16.
If BitsMachineID is 31 or more, an error is returned.
- TimeUnit is the time unit of Sonyflake.
If TimeUnit is 0, the default time unit is used, which is 10 msec.
TimeUnit must be 1 msec or longer.
- StartTime is the time since which the Sonyflake time is defined as the elapsed time.
If StartTime is 0, the start time of the Sonyflake is set to "2014-09-01 00:00:00 +0000 UTC".
If StartTime is ahead of the current time, Sonyflake is not created.
If StartTime is 0, the start time of the Sonyflake instance is set to "2025-01-01 00:00:00 +0000 UTC".
StartTime must be before the current time.
- MachineID returns the unique ID of the Sonyflake instance.
If MachineID returns an error, Sonyflake is not created.
If MachineID is nil, default MachineID is used.
Default MachineID returns the lower 16 bits of the private IP address.
- MachineID returns the unique ID of a Sonyflake instance.
If MachineID returns an error, the instance will not be created.
If MachineID is nil, the default MachineID is used, which returns the lower 16 bits of the private IP address.
- CheckMachineID validates the uniqueness of the machine ID.
If CheckMachineID returns false, Sonyflake is not created.
- CheckMachineID validates the uniqueness of a machine ID.
If CheckMachineID returns false, the instance will not be created.
If CheckMachineID is nil, no validation is done.
The bit length of time is calculated by 63 - BitsSequence - BitsMachineID.
If it is less than 32, an error is returned.
In order to get a new unique ID, you just have to call the method NextID.
```go
func (sf *Sonyflake) NextID() (uint64, error)
func (sf *Sonyflake) NextID() (int64, error)
```
NextID can continue to generate IDs for about 174 years from StartTime.
NextID can continue to generate IDs for about 174 years from StartTime by default.
But after the Sonyflake time is over the limit, NextID returns an error.
AWS VPC and Docker
------------------
## AWS VPC and Docker
The [awsutil](https://github.com/sony/sonyflake/blob/master/awsutil) package provides
The [awsutil](https://github.com/sony/sonyflake/blob/master/v2/awsutil) package provides
the function AmazonEC2MachineID that returns the lower 16-bit private IP address of the Amazon EC2 instance.
It also works correctly on Docker
by retrieving [instance metadata](http://docs.aws.amazon.com/en_us/AWSEC2/latest/UserGuide/ec2-instance-metadata.html).
[AWS VPC](http://docs.aws.amazon.com/en_us/AmazonVPC/latest/UserGuide/VPC_Subnets.html)
is assigned a single CIDR with a netmask between /28 and /16.
[AWS IPv4 VPC](https://docs.aws.amazon.com/vpc/latest/userguide/vpc-cidr-blocks.html)
is usually assigned a single CIDR with a netmask between /28 and /16.
So if each EC2 instance has a unique private IP address in AWS VPC,
the lower 16 bits of the address is also unique.
In this common case, you can use AmazonEC2MachineID as Settings.MachineID.
See [example](https://github.com/sony/sonyflake/blob/master/example) that runs Sonyflake on AWS Elastic Beanstalk.
See [example](https://github.com/sony/sonyflake/blob/master/v2/example) that runs Sonyflake on AWS Elastic Beanstalk.
License
-------
## License
The MIT License (MIT)

View File

@@ -1,6 +1,9 @@
FROM ubuntu:14.04
ADD ./sonyflake_server /
ENTRYPOINT ["/sonyflake_server"]
COPY ./sonyflake_server /sonyflake_server
RUN useradd -m sonyflake
USER sonyflake
ENTRYPOINT ["/sonyflake_server"]
EXPOSE 8080

View File

@@ -1,21 +1,19 @@
Example
=======
# Example
This example runs Sonyflake on AWS Elastic Beanstalk.
Setup
-----
## Setup
1. Build the cross compiler for linux/amd64 if using other platforms.
```
cd $GOROOT/src && GOOS=linux GOARCH=amd64 ./make.bash
```
```bash
cd $GOROOT/src && GOOS=linux GOARCH=amd64 ./make.bash
```
2. Build sonyflake_server in the example directory.
```
./linux64_build.sh
```
```bash
./linux64_build.sh
```
3. Upload the example directory to AWS Elastic Beanstalk.

View File

@@ -33,10 +33,17 @@ func handler(w http.ResponseWriter, r *http.Request) {
}
w.Header()["Content-Type"] = []string{"application/json; charset=utf-8"}
w.Write(body)
_, err = w.Write(body)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
}
func main() {
http.HandleFunc("/", handler)
http.ListenAndServe(":8080", nil)
err := http.ListenAndServe(":8080", nil)
if err != nil {
panic(err)
}
}

4
go.mod
View File

@@ -1,5 +1,3 @@
module github.com/sony/sonyflake
go 1.12
require github.com/deckarep/golang-set v1.7.1
go 1.13

2
go.sum
View File

@@ -1,2 +0,0 @@
github.com/deckarep/golang-set v1.7.1 h1:SCQV0S6gTtp6itiFrTqI+pfmJ4LN85S1YzhDf9rTHJQ=
github.com/deckarep/golang-set v1.7.1/go.mod h1:93vsz/8Wt4joVM7c2AVqh+YRMiUSc14yDtF28KmMOgQ=

34
mock/mock.go Normal file
View File

@@ -0,0 +1,34 @@
// Package mock offers mock implementations of interfaces defined in types.go.
// This allows complete control over input / output for any given method that consumes a given type.
package mock
import (
"fmt"
"net"
"github.com/sony/sonyflake/types"
)
// NewSuccessfulInterfaceAddrs returns a single private IP address.
func NewSuccessfulInterfaceAddrs() types.InterfaceAddrs {
ifat := make([]net.Addr, 0, 1)
ifat = append(ifat, &net.IPNet{IP: []byte{192, 168, 0, 1}, Mask: []byte{255, 0, 0, 0}})
return func() ([]net.Addr, error) {
return ifat, nil
}
}
// NewFailingInterfaceAddrs returns an error.
func NewFailingInterfaceAddrs() types.InterfaceAddrs {
return func() ([]net.Addr, error) {
return nil, fmt.Errorf("test error")
}
}
// NewNilInterfaceAddrs returns an empty slice of addresses.
func NewNilInterfaceAddrs() types.InterfaceAddrs {
return func() ([]net.Addr, error) {
return []net.Addr{}, nil
}
}

View File

@@ -1,9 +1,10 @@
// Package sonyflake implements Sonyflake, a distributed unique ID generator inspired by Twitter's Snowflake.
//
// A Sonyflake ID is composed of
// 39 bits for time in units of 10 msec
// 8 bits for a sequence number
// 16 bits for a machine id
//
// 39 bits for time in units of 10 msec
// 8 bits for a sequence number
// 16 bits for a machine id
package sonyflake
import (
@@ -11,6 +12,8 @@ import (
"net"
"sync"
"time"
"github.com/sony/sonyflake/types"
)
// These constants are the bit lengths of Sonyflake ID parts.
@@ -49,19 +52,29 @@ type Sonyflake struct {
machineID uint16
}
// NewSonyflake returns a new Sonyflake configured with the given Settings.
// NewSonyflake returns nil in the following cases:
var (
ErrStartTimeAhead = errors.New("start time is ahead of now")
ErrNoPrivateAddress = errors.New("no private ip address")
ErrOverTimeLimit = errors.New("over the time limit")
ErrInvalidMachineID = errors.New("invalid machine id")
)
var defaultInterfaceAddrs = net.InterfaceAddrs
// New returns a new Sonyflake configured with the given Settings.
// New returns an error in the following cases:
// - Settings.StartTime is ahead of the current time.
// - Settings.MachineID returns an error.
// - Settings.CheckMachineID returns false.
func NewSonyflake(st Settings) *Sonyflake {
func New(st Settings) (*Sonyflake, error) {
if st.StartTime.After(time.Now()) {
return nil, ErrStartTimeAhead
}
sf := new(Sonyflake)
sf.mutex = new(sync.Mutex)
sf.sequence = uint16(1<<BitLenSequence - 1)
if st.StartTime.After(time.Now()) {
return nil
}
if st.StartTime.IsZero() {
sf.startTime = toSonyflakeTime(time.Date(2014, 9, 1, 0, 0, 0, 0, time.UTC))
} else {
@@ -70,14 +83,28 @@ func NewSonyflake(st Settings) *Sonyflake {
var err error
if st.MachineID == nil {
sf.machineID, err = lower16BitPrivateIP()
sf.machineID, err = lower16BitPrivateIP(defaultInterfaceAddrs)
} else {
sf.machineID, err = st.MachineID()
}
if err != nil || (st.CheckMachineID != nil && !st.CheckMachineID(sf.machineID)) {
return nil
if err != nil {
return nil, err
}
if st.CheckMachineID != nil && !st.CheckMachineID(sf.machineID) {
return nil, ErrInvalidMachineID
}
return sf, nil
}
// NewSonyflake returns a new Sonyflake configured with the given Settings.
// NewSonyflake returns nil in the following cases:
// - Settings.StartTime is ahead of the current time.
// - Settings.MachineID returns an error.
// - Settings.CheckMachineID returns false.
func NewSonyflake(st Settings) *Sonyflake {
sf, _ := New(st)
return sf
}
@@ -116,13 +143,13 @@ func currentElapsedTime(startTime int64) int64 {
}
func sleepTime(overtime int64) time.Duration {
return time.Duration(overtime)*10*time.Millisecond -
time.Duration(time.Now().UTC().UnixNano()%sonyflakeTimeUnit)*time.Nanosecond
return time.Duration(overtime*sonyflakeTimeUnit) -
time.Duration(time.Now().UTC().UnixNano()%sonyflakeTimeUnit)
}
func (sf *Sonyflake) toID() (uint64, error) {
if sf.elapsedTime >= 1<<BitLenTime {
return 0, errors.New("over the time limit")
return 0, ErrOverTimeLimit
}
return uint64(sf.elapsedTime)<<(BitLenSequence+BitLenMachineID) |
@@ -130,8 +157,8 @@ func (sf *Sonyflake) toID() (uint64, error) {
uint64(sf.machineID), nil
}
func privateIPv4() (net.IP, error) {
as, err := net.InterfaceAddrs()
func privateIPv4(interfaceAddrs types.InterfaceAddrs) (net.IP, error) {
as, err := interfaceAddrs()
if err != nil {
return nil, err
}
@@ -147,16 +174,17 @@ func privateIPv4() (net.IP, error) {
return ip, nil
}
}
return nil, errors.New("no private ip address")
return nil, ErrNoPrivateAddress
}
func isPrivateIPv4(ip net.IP) bool {
// Allow private IP addresses (RFC1918) and link-local addresses (RFC3927)
return ip != nil &&
(ip[0] == 10 || ip[0] == 172 && (ip[1] >= 16 && ip[1] < 32) || ip[0] == 192 && ip[1] == 168)
(ip[0] == 10 || ip[0] == 172 && (ip[1] >= 16 && ip[1] < 32) || ip[0] == 192 && ip[1] == 168 || ip[0] == 169 && ip[1] == 254)
}
func lower16BitPrivateIP() (uint16, error) {
ip, err := privateIPv4()
func lower16BitPrivateIP(interfaceAddrs types.InterfaceAddrs) (uint16, error) {
ip, err := privateIPv4(interfaceAddrs)
if err != nil {
return 0, err
}
@@ -164,15 +192,33 @@ func lower16BitPrivateIP() (uint16, error) {
return uint16(ip[2])<<8 + uint16(ip[3]), nil
}
// ElapsedTime returns the elapsed time when the given Sonyflake ID was generated.
func ElapsedTime(id uint64) time.Duration {
return time.Duration(elapsedTime(id) * sonyflakeTimeUnit)
}
func elapsedTime(id uint64) uint64 {
return id >> (BitLenSequence + BitLenMachineID)
}
// SequenceNumber returns the sequence number of a Sonyflake ID.
func SequenceNumber(id uint64) uint64 {
const maskSequence = uint64((1<<BitLenSequence - 1) << BitLenMachineID)
return id & maskSequence >> BitLenMachineID
}
// MachineID returns the machine ID of a Sonyflake ID.
func MachineID(id uint64) uint64 {
const maskMachineID = uint64(1<<BitLenMachineID - 1)
return id & maskMachineID
}
// Decompose returns a set of Sonyflake ID parts.
func Decompose(id uint64) map[string]uint64 {
const maskSequence = uint64((1<<BitLenSequence - 1) << BitLenMachineID)
const maskMachineID = uint64(1<<BitLenMachineID - 1)
msb := id >> 63
time := id >> (BitLenSequence + BitLenMachineID)
sequence := id & maskSequence >> BitLenMachineID
machineID := id & maskMachineID
time := elapsedTime(id)
sequence := SequenceNumber(id)
machineID := MachineID(id)
return map[string]uint64{
"id": id,
"msb": msb,

View File

@@ -1,12 +1,15 @@
package sonyflake
import (
"errors"
"fmt"
"net"
"runtime"
"testing"
"time"
"github.com/deckarep/golang-set"
"github.com/sony/sonyflake/mock"
"github.com/sony/sonyflake/types"
)
var sf *Sonyflake
@@ -25,7 +28,7 @@ func init() {
startTime = toSonyflakeTime(st.StartTime)
ip, _ := lower16BitPrivateIP()
ip, _ := lower16BitPrivateIP(defaultInterfaceAddrs)
machineID = uint64(ip)
}
@@ -37,35 +40,83 @@ func nextID(t *testing.T) uint64 {
return id
}
func TestSonyflakeOnce(t *testing.T) {
sleepTime := uint64(50)
time.Sleep(time.Duration(sleepTime) * 10 * time.Millisecond)
func TestNew(t *testing.T) {
genError := fmt.Errorf("an error occurred while generating ID")
id := nextID(t)
parts := Decompose(id)
actualMSB := parts["msb"]
if actualMSB != 0 {
t.Errorf("unexpected msb: %d", actualMSB)
tests := []struct {
name string
settings Settings
err error
}{
{
name: "failure: time ahead",
settings: Settings{
StartTime: time.Now().Add(time.Minute),
},
err: ErrStartTimeAhead,
},
{
name: "failure: machine ID",
settings: Settings{
MachineID: func() (uint16, error) {
return 0, genError
},
},
err: genError,
},
{
name: "failure: invalid machine ID",
settings: Settings{
CheckMachineID: func(uint16) bool {
return false
},
},
err: ErrInvalidMachineID,
},
{
name: "success",
settings: Settings{},
},
}
actualTime := parts["time"]
if actualTime < sleepTime || actualTime > sleepTime+1 {
for _, test := range tests {
t.Run(test.name, func(t *testing.T) {
sonyflake, err := New(test.settings)
if !errors.Is(err, test.err) {
t.Fatalf("unexpected value, want %#v, got %#v", test.err, err)
}
if sonyflake == nil && err == nil {
t.Fatal("unexpected value, sonyflake should not be nil")
}
})
}
}
func TestSonyflakeOnce(t *testing.T) {
sleepTime := time.Duration(50 * sonyflakeTimeUnit)
time.Sleep(sleepTime)
id := nextID(t)
actualTime := ElapsedTime(id)
if actualTime < sleepTime || actualTime > sleepTime+sonyflakeTimeUnit {
t.Errorf("unexpected time: %d", actualTime)
}
actualSequence := parts["sequence"]
actualSequence := SequenceNumber(id)
if actualSequence != 0 {
t.Errorf("unexpected sequence: %d", actualSequence)
}
actualMachineID := parts["machine-id"]
actualMachineID := MachineID(id)
if actualMachineID != machineID {
t.Errorf("unexpected machine id: %d", actualMachineID)
}
fmt.Println("sonyflake id:", id)
fmt.Println("decompose:", parts)
fmt.Println("decompose:", Decompose(id))
}
func currentTime() int64 {
@@ -84,9 +135,12 @@ func TestSonyflakeFor10Sec(t *testing.T) {
parts := Decompose(id)
numID++
if id <= lastID {
if id == lastID {
t.Fatal("duplicated id")
}
if id < lastID {
t.Fatal("must increase with time")
}
lastID = id
current = currentTime()
@@ -139,40 +193,15 @@ func TestSonyflakeInParallel(t *testing.T) {
go generate()
}
set := mapset.NewSet()
set := make(map[uint64]struct{})
for i := 0; i < numID*numGenerator; i++ {
id := <-consumer
if set.Contains(id) {
if _, ok := set[id]; ok {
t.Fatal("duplicated id")
} else {
set.Add(id)
}
set[id] = struct{}{}
}
fmt.Println("number of id:", set.Cardinality())
}
func TestNilSonyflake(t *testing.T) {
var startInFuture Settings
startInFuture.StartTime = time.Now().Add(time.Duration(1) * time.Minute)
if NewSonyflake(startInFuture) != nil {
t.Errorf("sonyflake starting in the future")
}
var noMachineID Settings
noMachineID.MachineID = func() (uint16, error) {
return 0, fmt.Errorf("no machine id")
}
if NewSonyflake(noMachineID) != nil {
t.Errorf("sonyflake with no machine id")
}
var invalidMachineID Settings
invalidMachineID.CheckMachineID = func(uint16) bool {
return false
}
if NewSonyflake(invalidMachineID) != nil {
t.Errorf("sonyflake with invalid machine id")
}
fmt.Println("number of id:", len(set))
}
func pseudoSleep(period time.Duration) {
@@ -190,3 +219,96 @@ func TestNextIDError(t *testing.T) {
t.Errorf("time is not over")
}
}
func TestPrivateIPv4(t *testing.T) {
testCases := []struct {
description string
expected net.IP
interfaceAddrs types.InterfaceAddrs
error string
}{
{
description: "InterfaceAddrs returns an error",
expected: nil,
interfaceAddrs: mock.NewFailingInterfaceAddrs(),
error: "test error",
},
{
description: "InterfaceAddrs returns an empty or nil list",
expected: nil,
interfaceAddrs: mock.NewNilInterfaceAddrs(),
error: "no private ip address",
},
{
description: "InterfaceAddrs returns one or more IPs",
expected: net.IP{192, 168, 0, 1},
interfaceAddrs: mock.NewSuccessfulInterfaceAddrs(),
error: "",
},
}
for _, tc := range testCases {
t.Run(tc.description, func(t *testing.T) {
actual, err := privateIPv4(tc.interfaceAddrs)
if (err != nil) && (tc.error == "") {
t.Errorf("expected no error, but got: %s", err)
return
} else if (err != nil) && (tc.error != "") {
return
}
if net.IP.Equal(actual, tc.expected) {
return
} else {
t.Errorf("error: expected: %s, but got: %s", tc.expected, actual)
}
})
}
}
func TestLower16BitPrivateIP(t *testing.T) {
testCases := []struct {
description string
expected uint16
interfaceAddrs types.InterfaceAddrs
error string
}{
{
description: "InterfaceAddrs returns an empty or nil list",
expected: 0,
interfaceAddrs: mock.NewNilInterfaceAddrs(),
error: "no private ip address",
},
{
description: "InterfaceAddrs returns one or more IPs",
expected: 1,
interfaceAddrs: mock.NewSuccessfulInterfaceAddrs(),
error: "",
},
}
for _, tc := range testCases {
t.Run(tc.description, func(t *testing.T) {
actual, err := lower16BitPrivateIP(tc.interfaceAddrs)
if (err != nil) && (tc.error == "") {
t.Errorf("expected no error, but got: %s", err)
return
} else if (err != nil) && (tc.error != "") {
return
}
if actual == tc.expected {
return
} else {
t.Errorf("error: expected: %v, but got: %v", tc.expected, actual)
}
})
}
}
func TestSonyflakeTimeUnit(t *testing.T) {
if time.Duration(sonyflakeTimeUnit) != 10*time.Millisecond {
t.Errorf("unexpected time unit")
}
}

8
types/types.go Normal file
View File

@@ -0,0 +1,8 @@
// Package types defines type signatures used throughout sonyflake.
// This provides the ability to mock out imports.
package types
import "net"
// InterfaceAddrs defines the interface used for retrieving network addresses.
type InterfaceAddrs func() ([]net.Addr, error)

64
v2/awsutil/awsutil.go Normal file
View File

@@ -0,0 +1,64 @@
// Package awsutil provides utility functions for using Sonyflake on AWS.
package awsutil
import (
"errors"
"io"
"net"
"net/http"
"os/exec"
"regexp"
"strconv"
"time"
)
func amazonEC2PrivateIPv4() (net.IP, error) {
res, err := http.Get("http://169.254.169.254/latest/meta-data/local-ipv4")
if err != nil {
return nil, err
}
defer res.Body.Close()
body, err := io.ReadAll(res.Body)
if err != nil {
return nil, err
}
ip := net.ParseIP(string(body))
if ip == nil {
return nil, errors.New("invalid ip address")
}
return ip.To4(), nil
}
// AmazonEC2MachineID retrieves the private IP address of the Amazon EC2 instance
// and returns its lower 16 bits.
// It works correctly on Docker as well.
func AmazonEC2MachineID() (int, error) {
ip, err := amazonEC2PrivateIPv4()
if err != nil {
return 0, err
}
return int(ip[2])<<8 + int(ip[3]), nil
}
// TimeDifference returns the time difference between the localhost and the given NTP server.
func TimeDifference(server string) (time.Duration, error) {
output, err := exec.Command("/usr/sbin/ntpdate", "-q", server).CombinedOutput()
if err != nil {
return time.Duration(0), err
}
re, _ := regexp.Compile("offset (.*) sec")
submatched := re.FindSubmatch(output)
if len(submatched) != 2 {
return time.Duration(0), errors.New("invalid ntpdate output")
}
f, err := strconv.ParseFloat(string(submatched[1]), 64)
if err != nil {
return time.Duration(0), err
}
return time.Duration(f*1000) * time.Millisecond, nil
}

9
v2/example/Dockerfile Normal file
View File

@@ -0,0 +1,9 @@
FROM ubuntu:24.04
COPY ./sonyflake_server /sonyflake_server
RUN useradd -m sonyflake
USER sonyflake
ENTRYPOINT ["/sonyflake_server"]
EXPOSE 8080

View File

@@ -0,0 +1,8 @@
{
"AWSEBDockerrunVersion": "1",
"Ports": [
{
"ContainerPort": "8080"
}
]
}

19
v2/example/README.md Normal file
View File

@@ -0,0 +1,19 @@
# Example
This example runs Sonyflake on AWS Elastic Beanstalk.
## Setup
1. Build the cross compiler for linux/amd64 if using other platforms.
```bash
cd $GOROOT/src && GOOS=linux GOARCH=amd64 ./make.bash
```
2. Build sonyflake_server in the example directory.
```bash
./linux64_build.sh
```
3. Upload the example directory to AWS Elastic Beanstalk.

2
v2/example/linux64_build.sh Executable file
View File

@@ -0,0 +1,2 @@
#!/bin/sh
GOOS=linux GOARCH=amd64 go build sonyflake_server.go

View File

@@ -0,0 +1,51 @@
package main
import (
"encoding/json"
"net/http"
"github.com/sony/sonyflake/v2"
"github.com/sony/sonyflake/v2/awsutil"
)
var sf *sonyflake.Sonyflake
func init() {
var st sonyflake.Settings
st.MachineID = awsutil.AmazonEC2MachineID
var err error
sf, err = sonyflake.New(st)
if err != nil {
panic(err)
}
}
func handler(w http.ResponseWriter, r *http.Request) {
id, err := sf.NextID()
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
body, err := json.Marshal(sf.Decompose(id))
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
w.Header()["Content-Type"] = []string{"application/json; charset=utf-8"}
_, err = w.Write(body)
if err != nil {
http.Error(w, err.Error(), http.StatusInternalServerError)
return
}
}
func main() {
http.HandleFunc("/", handler)
err := http.ListenAndServe(":8080", nil)
if err != nil {
panic(err)
}
}

3
v2/go.mod Normal file
View File

@@ -0,0 +1,3 @@
module github.com/sony/sonyflake/v2
go 1.22

36
v2/mock/mock.go Normal file
View File

@@ -0,0 +1,36 @@
// Package mock offers mock implementations of interfaces defined in types.go.
// This allows complete control over input / output for any given method that consumes a given type.
package mock
import (
"errors"
"net"
"github.com/sony/sonyflake/v2/types"
)
// NewSuccessfulInterfaceAddrs returns a single private IP address.
func NewSuccessfulInterfaceAddrs() types.InterfaceAddrs {
ifat := make([]net.Addr, 0, 1)
ifat = append(ifat, &net.IPNet{IP: []byte{192, 168, 0, 1}, Mask: []byte{255, 0, 0, 0}})
return func() ([]net.Addr, error) {
return ifat, nil
}
}
var ErrFailedToGetAddresses = errors.New("failed to get addresses")
// NewFailingInterfaceAddrs returns an error.
func NewFailingInterfaceAddrs() types.InterfaceAddrs {
return func() ([]net.Addr, error) {
return nil, ErrFailedToGetAddresses
}
}
// NewNilInterfaceAddrs returns an empty slice of addresses.
func NewNilInterfaceAddrs() types.InterfaceAddrs {
return func() ([]net.Addr, error) {
return []net.Addr{}, nil
}
}

311
v2/sonyflake.go Normal file
View File

@@ -0,0 +1,311 @@
// Package sonyflake implements Sonyflake, a distributed unique ID generator inspired by Twitter's Snowflake.
//
// By default, a Sonyflake ID is composed of
//
// 39 bits for time in units of 10 msec
// 8 bits for a sequence number
// 16 bits for a machine id
package sonyflake
import (
"errors"
"net"
"sync"
"time"
"github.com/sony/sonyflake/v2/types"
)
// Settings configures Sonyflake:
//
// BitsSequence is the bit length of a sequence number.
// If BitsSequence is 0, the default bit length is used, which is 8.
// If BitsSequence is 31 or more, an error is returned.
//
// BitsMachineID is the bit length of a machine ID.
// If BitsMachineID is 0, the default bit length is used, which is 16.
// If BitsMachineID is 31 or more, an error is returned.
//
// TimeUnit is the time unit of Sonyflake.
// If TimeUnit is 0, the default time unit is used, which is 10 msec.
// TimeUnit must be 1 msec or longer.
//
// StartTime is the time since which the Sonyflake time is defined as the elapsed time.
// If StartTime is 0, the start time of the Sonyflake instance is set to "2025-01-01 00:00:00 +0000 UTC".
// StartTime must be before the current time.
//
// MachineID returns the unique ID of a Sonyflake instance.
// If MachineID returns an error, the instance will not be created.
// If MachineID is nil, the default MachineID is used, which returns the lower 16 bits of the private IP address.
//
// CheckMachineID validates the uniqueness of a machine ID.
// If CheckMachineID returns false, the instance will not be created.
// If CheckMachineID is nil, no validation is done.
//
// The bit length of time is calculated by 63 - BitsSequence - BitsMachineID.
// If it is less than 32, an error is returned.
type Settings struct {
BitsSequence int
BitsMachineID int
TimeUnit time.Duration
StartTime time.Time
MachineID func() (int, error)
CheckMachineID func(int) bool
}
// Sonyflake is a distributed unique ID generator.
type Sonyflake struct {
mutex *sync.Mutex
bitsTime int
bitsSequence int
bitsMachine int
timeUnit int64
startTime int64
elapsedTime int64
sequence int
machine int
}
var (
ErrInvalidBitsTime = errors.New("bit length for time must be 32 or more")
ErrInvalidBitsSequence = errors.New("invalid bit length for sequence number")
ErrInvalidBitsMachineID = errors.New("invalid bit length for machine id")
ErrInvalidTimeUnit = errors.New("invalid time unit")
ErrInvalidSequence = errors.New("invalid sequence number")
ErrInvalidMachineID = errors.New("invalid machine id")
ErrStartTimeAhead = errors.New("start time is ahead")
ErrOverTimeLimit = errors.New("over the time limit")
ErrNoPrivateAddress = errors.New("no private ip address")
)
const (
defaultTimeUnit = 1e7 // nsec, i.e. 10 msec
defaultBitsTime = 39
defaultBitsSequence = 8
defaultBitsMachine = 16
)
var defaultInterfaceAddrs = net.InterfaceAddrs
// New returns a new Sonyflake configured with the given Settings.
// New returns an error in the following cases:
// - Settings.BitsSequence is less than 0 or greater than 30.
// - Settings.BitsMachineID is less than 0 or greater than 30.
// - Settings.BitsSequence + Settings.BitsMachineID is 32 or more.
// - Settings.TimeUnit is less than 1 msec.
// - Settings.StartTime is ahead of the current time.
// - Settings.MachineID returns an error.
// - Settings.CheckMachineID returns false.
func New(st Settings) (*Sonyflake, error) {
if st.BitsSequence < 0 || st.BitsSequence > 30 {
return nil, ErrInvalidBitsSequence
}
if st.BitsMachineID < 0 || st.BitsMachineID > 30 {
return nil, ErrInvalidBitsMachineID
}
if st.TimeUnit < 0 || (st.TimeUnit > 0 && st.TimeUnit < time.Millisecond) {
return nil, ErrInvalidTimeUnit
}
if st.StartTime.After(time.Now()) {
return nil, ErrStartTimeAhead
}
sf := new(Sonyflake)
sf.mutex = new(sync.Mutex)
if st.BitsSequence == 0 {
sf.bitsSequence = defaultBitsSequence
} else {
sf.bitsSequence = st.BitsSequence
}
if st.BitsMachineID == 0 {
sf.bitsMachine = defaultBitsMachine
} else {
sf.bitsMachine = st.BitsMachineID
}
sf.bitsTime = 63 - sf.bitsSequence - sf.bitsMachine
if sf.bitsTime < 32 {
return nil, ErrInvalidBitsTime
}
if st.TimeUnit == 0 {
sf.timeUnit = defaultTimeUnit
} else {
sf.timeUnit = int64(st.TimeUnit)
}
if st.StartTime.IsZero() {
sf.startTime = sf.toInternalTime(time.Date(2025, 1, 1, 0, 0, 0, 0, time.UTC))
} else {
sf.startTime = sf.toInternalTime(st.StartTime)
}
sf.sequence = 1<<sf.bitsSequence - 1
var err error
if st.MachineID == nil {
sf.machine, err = lower16BitPrivateIP(defaultInterfaceAddrs)
} else {
sf.machine, err = st.MachineID()
}
if err != nil {
return nil, err
}
if sf.machine < 0 || sf.machine >= 1<<sf.bitsMachine {
return nil, ErrInvalidMachineID
}
if st.CheckMachineID != nil && !st.CheckMachineID(sf.machine) {
return nil, ErrInvalidMachineID
}
return sf, nil
}
// NextID generates a next unique ID as int64.
// After the Sonyflake time overflows, NextID returns an error.
func (sf *Sonyflake) NextID() (int64, error) {
maskSequence := 1<<sf.bitsSequence - 1
sf.mutex.Lock()
defer sf.mutex.Unlock()
current := sf.currentElapsedTime()
if sf.elapsedTime < current {
sf.elapsedTime = current
sf.sequence = 0
} else {
sf.sequence = (sf.sequence + 1) & maskSequence
if sf.sequence == 0 {
sf.elapsedTime++
overtime := sf.elapsedTime - current
sf.sleep(overtime)
}
}
return sf.toID()
}
func (sf *Sonyflake) toInternalTime(t time.Time) int64 {
return t.UTC().UnixNano() / sf.timeUnit
}
func (sf *Sonyflake) currentElapsedTime() int64 {
return sf.toInternalTime(time.Now()) - sf.startTime
}
func (sf *Sonyflake) sleep(overtime int64) {
sleepTime := time.Duration(overtime*sf.timeUnit) -
time.Duration(time.Now().UTC().UnixNano()%sf.timeUnit)
time.Sleep(sleepTime)
}
func (sf *Sonyflake) toID() (int64, error) {
if sf.elapsedTime >= 1<<sf.bitsTime {
return 0, ErrOverTimeLimit
}
return sf.elapsedTime<<(sf.bitsSequence+sf.bitsMachine) |
int64(sf.sequence)<<sf.bitsMachine |
int64(sf.machine), nil
}
func privateIPv4(interfaceAddrs types.InterfaceAddrs) (net.IP, error) {
as, err := interfaceAddrs()
if err != nil {
return nil, err
}
for _, a := range as {
ipnet, ok := a.(*net.IPNet)
if !ok || ipnet.IP.IsLoopback() {
continue
}
ip := ipnet.IP.To4()
if isPrivateIPv4(ip) {
return ip, nil
}
}
return nil, ErrNoPrivateAddress
}
func isPrivateIPv4(ip net.IP) bool {
// Allow private IP addresses (RFC1918) and link-local addresses (RFC3927)
return ip != nil &&
(ip[0] == 10 || ip[0] == 172 && (ip[1] >= 16 && ip[1] < 32) || ip[0] == 192 && ip[1] == 168 || ip[0] == 169 && ip[1] == 254)
}
func lower16BitPrivateIP(interfaceAddrs types.InterfaceAddrs) (int, error) {
ip, err := privateIPv4(interfaceAddrs)
if err != nil {
return 0, err
}
return int(ip[2])<<8 + int(ip[3]), nil
}
// ToTime returns the time when the given ID was generated.
func (sf *Sonyflake) ToTime(id int64) time.Time {
return time.Unix(0, (sf.startTime+sf.timePart(id))*sf.timeUnit)
}
// Compose creates a Sonyflake ID from its components.
// The time parameter should be the time when the ID was generated.
// The sequence parameter should be between 0 and 2^BitsSequence-1 (inclusive).
// The machineID parameter should be between 0 and 2^BitsMachineID-1 (inclusive).
func (sf *Sonyflake) Compose(t time.Time, sequence, machineID int) (int64, error) {
elapsedTime := sf.toInternalTime(t.UTC()) - sf.startTime
if elapsedTime < 0 {
return 0, ErrStartTimeAhead
}
if elapsedTime >= 1<<sf.bitsTime {
return 0, ErrOverTimeLimit
}
if sequence < 0 || sequence >= 1<<sf.bitsSequence {
return 0, ErrInvalidSequence
}
if machineID < 0 || machineID >= 1<<sf.bitsMachine {
return 0, ErrInvalidMachineID
}
return elapsedTime<<(sf.bitsSequence+sf.bitsMachine) |
int64(sequence)<<sf.bitsMachine |
int64(machineID), nil
}
// Decompose returns a set of Sonyflake ID parts.
func (sf *Sonyflake) Decompose(id int64) map[string]int64 {
time := sf.timePart(id)
sequence := sf.sequencePart(id)
machine := sf.machinePart(id)
return map[string]int64{
"id": id,
"time": time,
"sequence": sequence,
"machine": machine,
}
}
func (sf *Sonyflake) timePart(id int64) int64 {
return id >> (sf.bitsSequence + sf.bitsMachine)
}
func (sf *Sonyflake) sequencePart(id int64) int64 {
maskSequence := int64((1<<sf.bitsSequence - 1) << sf.bitsMachine)
return (id & maskSequence) >> sf.bitsMachine
}
func (sf *Sonyflake) machinePart(id int64) int64 {
maskMachine := int64(1<<sf.bitsMachine - 1)
return id & maskMachine
}

495
v2/sonyflake_test.go Normal file
View File

@@ -0,0 +1,495 @@
package sonyflake
import (
"errors"
"fmt"
"net"
"runtime"
"testing"
"time"
"github.com/sony/sonyflake/v2/mock"
"github.com/sony/sonyflake/v2/types"
)
func TestNew(t *testing.T) {
errGetMachineID := fmt.Errorf("failed to get machine id")
testCases := []struct {
name string
settings Settings
err error
}{
{
name: "invalid bit length for time",
settings: Settings{
BitsSequence: 16,
BitsMachineID: 16,
},
err: ErrInvalidBitsTime,
},
{
name: "invalid bit length for sequence number",
settings: Settings{
BitsSequence: -1,
},
err: ErrInvalidBitsSequence,
},
{
name: "invalid bit length for machine id",
settings: Settings{
BitsMachineID: 31,
},
err: ErrInvalidBitsMachineID,
},
{
name: "invalid time unit",
settings: Settings{
TimeUnit: time.Microsecond,
},
err: ErrInvalidTimeUnit,
},
{
name: "start time ahead",
settings: Settings{
StartTime: time.Now().Add(time.Minute),
},
err: ErrStartTimeAhead,
},
{
name: "cannot get machine id",
settings: Settings{
MachineID: func() (int, error) {
return 0, errGetMachineID
},
},
err: errGetMachineID,
},
{
name: "too large machine id",
settings: Settings{
MachineID: func() (int, error) {
return 1 << defaultBitsMachine, nil
},
},
err: ErrInvalidMachineID,
},
{
name: "negative machine id",
settings: Settings{
MachineID: func() (int, error) {
return -1, nil
},
},
err: ErrInvalidMachineID,
},
{
name: "invalid machine id",
settings: Settings{
CheckMachineID: func(int) bool {
return false
},
},
err: ErrInvalidMachineID,
},
{
name: "success",
settings: Settings{},
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
sf, err := New(tc.settings)
if !errors.Is(err, tc.err) {
t.Fatalf("unexpected error: %v", err)
}
if err == nil && sf == nil {
t.Fatal("sonyflake instance must be created")
}
})
}
}
func newSonyflake(t *testing.T, st Settings) *Sonyflake {
sf, err := New(st)
if err != nil {
t.Fatalf("failed to create sonyflake: %v", err)
}
return sf
}
func nextID(t *testing.T, sf *Sonyflake) int64 {
id, err := sf.NextID()
if err != nil {
t.Fatalf("failed to generate id: %v", err)
}
return id
}
func defaultMachineID(t *testing.T) int {
ip, err := lower16BitPrivateIP(defaultInterfaceAddrs)
if err != nil {
t.Fatalf("failed to get private ip address: %v", err)
}
return ip
}
func TestNextID(t *testing.T) {
sf := newSonyflake(t, Settings{StartTime: time.Now()})
sleepTime := int64(50)
time.Sleep(time.Duration(sleepTime * sf.timeUnit))
id := nextID(t, sf)
actualTime := sf.timePart(id)
if actualTime < sleepTime || actualTime > sleepTime+1 {
t.Errorf("unexpected time: %d", actualTime)
}
actualSequence := sf.sequencePart(id)
if actualSequence != 0 {
t.Errorf("unexpected sequence: %d", actualSequence)
}
actualMachine := sf.machinePart(id)
if actualMachine != int64(defaultMachineID(t)) {
t.Errorf("unexpected machine: %d", actualMachine)
}
fmt.Println("sonyflake id:", id)
fmt.Println("decompose:", sf.Decompose(id))
}
func TestNextID_InSequence(t *testing.T) {
now := time.Now()
sf := newSonyflake(t, Settings{
TimeUnit: time.Millisecond,
StartTime: now,
})
startTime := sf.toInternalTime(now)
machineID := int64(defaultMachineID(t))
var numID int
var lastID int64
var maxSeq int64
currentTime := startTime
for currentTime-startTime < 100 {
id := nextID(t, sf)
currentTime = sf.toInternalTime(time.Now())
numID++
if id == lastID {
t.Fatal("duplicated id")
}
if id < lastID {
t.Fatal("must increase with time")
}
lastID = id
parts := sf.Decompose(id)
actualTime := parts["time"]
overtime := startTime + actualTime - currentTime
if overtime > 0 {
t.Errorf("unexpected overtime: %d", overtime)
}
actualSequence := parts["sequence"]
if actualSequence > maxSeq {
maxSeq = actualSequence
}
actualMachine := parts["machine"]
if actualMachine != machineID {
t.Errorf("unexpected machine: %d", actualMachine)
}
}
if maxSeq != 1<<sf.bitsSequence-1 {
t.Errorf("unexpected max sequence: %d", maxSeq)
}
fmt.Println("max sequence:", maxSeq)
fmt.Println("number of id:", numID)
}
func TestNextID_InParallel(t *testing.T) {
sf1 := newSonyflake(t, Settings{MachineID: func() (int, error) { return 1, nil }})
sf2 := newSonyflake(t, Settings{MachineID: func() (int, error) { return 2, nil }})
numCPU := runtime.NumCPU()
runtime.GOMAXPROCS(numCPU)
fmt.Println("number of cpu:", numCPU)
consumer := make(chan int64)
const numID = 1000
generate := func(sf *Sonyflake) {
for i := 0; i < numID; i++ {
id := nextID(t, sf)
consumer <- id
}
}
var numGenerator int
for i := 0; i < numCPU/2; i++ {
go generate(sf1)
go generate(sf2)
numGenerator += 2
}
set := make(map[int64]struct{})
for i := 0; i < numID*numGenerator; i++ {
id := <-consumer
if _, ok := set[id]; ok {
t.Fatal("duplicated id")
}
set[id] = struct{}{}
}
fmt.Println("number of id:", len(set))
}
func pseudoSleep(sf *Sonyflake, period time.Duration) {
sf.startTime -= int64(period) / sf.timeUnit
}
const year = time.Duration(365*24) * time.Hour
func TestNextID_ReturnsError(t *testing.T) {
sf := newSonyflake(t, Settings{StartTime: time.Now()})
pseudoSleep(sf, time.Duration(174)*year)
nextID(t, sf)
pseudoSleep(sf, time.Duration(1)*year)
_, err := sf.NextID()
if err == nil {
t.Errorf("time is not over")
}
}
func TestPrivateIPv4(t *testing.T) {
testCases := []struct {
description string
expected net.IP
interfaceAddrs types.InterfaceAddrs
error error
}{
{
description: "returns an error",
expected: nil,
interfaceAddrs: mock.NewFailingInterfaceAddrs(),
error: mock.ErrFailedToGetAddresses,
},
{
description: "empty address list",
expected: nil,
interfaceAddrs: mock.NewNilInterfaceAddrs(),
error: ErrNoPrivateAddress,
},
{
description: "success",
expected: net.IP{192, 168, 0, 1},
interfaceAddrs: mock.NewSuccessfulInterfaceAddrs(),
error: nil,
},
}
for _, tc := range testCases {
t.Run(tc.description, func(t *testing.T) {
actual, err := privateIPv4(tc.interfaceAddrs)
if !errors.Is(err, tc.error) {
t.Fatalf("unexpected error: %v", err)
}
if !net.IP.Equal(actual, tc.expected) {
t.Errorf("unexpected ip: %s", actual)
}
})
}
}
func TestLower16BitPrivateIP(t *testing.T) {
testCases := []struct {
description string
expected int
interfaceAddrs types.InterfaceAddrs
error error
}{
{
description: "returns an error",
expected: 0,
interfaceAddrs: mock.NewFailingInterfaceAddrs(),
error: mock.ErrFailedToGetAddresses,
},
{
description: "empty address list",
expected: 0,
interfaceAddrs: mock.NewNilInterfaceAddrs(),
error: ErrNoPrivateAddress,
},
{
description: "success",
expected: 1,
interfaceAddrs: mock.NewSuccessfulInterfaceAddrs(),
error: nil,
},
}
for _, tc := range testCases {
t.Run(tc.description, func(t *testing.T) {
actual, err := lower16BitPrivateIP(tc.interfaceAddrs)
if !errors.Is(err, tc.error) {
t.Fatalf("unexpected error: %v", err)
}
if actual != tc.expected {
t.Errorf("unexpected ip: %d", actual)
}
})
}
}
func TestToTime(t *testing.T) {
start := time.Now()
sf := newSonyflake(t, Settings{
TimeUnit: time.Millisecond,
StartTime: start,
})
id := nextID(t, sf)
tm := sf.ToTime(id)
diff := tm.Sub(start)
if diff < 0 || diff >= time.Duration(sf.timeUnit) {
t.Errorf("unexpected time: %v", tm)
}
}
func TestComposeAndDecompose(t *testing.T) {
now := time.Now()
sf := newSonyflake(t, Settings{
TimeUnit: time.Millisecond,
StartTime: now,
})
testCases := []struct {
name string
time time.Time
sequence int
machineID int
}{
{
name: "zero values",
time: now,
sequence: 0,
machineID: 0,
},
{
name: "max sequence",
time: now,
sequence: 1<<sf.bitsSequence - 1,
machineID: 0,
},
{
name: "max machine id",
time: now,
sequence: 0,
machineID: 1<<sf.bitsMachine - 1,
},
{
name: "future time",
time: now.Add(time.Hour),
sequence: 0,
machineID: 0,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
id, err := sf.Compose(tc.time, tc.sequence, tc.machineID)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
parts := sf.Decompose(id)
// Verify time part
expectedTime := sf.toInternalTime(tc.time.UTC()) - sf.startTime
if parts["time"] != expectedTime {
t.Errorf("time mismatch: got %d, want %d", parts["time"], expectedTime)
}
// Verify sequence part
if parts["sequence"] != int64(tc.sequence) {
t.Errorf("sequence mismatch: got %d, want %d", parts["sequence"], tc.sequence)
}
// Verify machine id part
if parts["machine"] != int64(tc.machineID) {
t.Errorf("machine id mismatch: got %d, want %d", parts["machine"], tc.machineID)
}
// Verify id part
if parts["id"] != id {
t.Errorf("id mismatch: got %d, want %d", parts["id"], id)
}
})
}
}
func TestCompose_ReturnsError(t *testing.T) {
start := time.Now()
sf := newSonyflake(t, Settings{StartTime: start})
testCases := []struct {
name string
time time.Time
sequence int
machineID int
err error
}{
{
name: "start time ahead",
time: start.Add(-time.Second),
sequence: 0,
machineID: 0,
err: ErrStartTimeAhead,
},
{
name: "over time limit",
time: start.Add(time.Duration(175) * year),
sequence: 0,
machineID: 0,
err: ErrOverTimeLimit,
},
{
name: "invalid sequence",
time: start,
sequence: 1 << sf.bitsSequence,
machineID: 0,
err: ErrInvalidSequence,
},
{
name: "invalid machine id",
time: start,
sequence: 0,
machineID: 1 << sf.bitsMachine,
err: ErrInvalidMachineID,
},
}
for _, tc := range testCases {
t.Run(tc.name, func(t *testing.T) {
_, err := sf.Compose(tc.time, tc.sequence, tc.machineID)
if !errors.Is(err, tc.err) {
t.Errorf("unexpected error: %v", err)
}
})
}
}

8
v2/types/types.go Normal file
View File

@@ -0,0 +1,8 @@
// Package types defines type signatures used throughout sonyflake.
// This provides the ability to mock out imports.
package types
import "net"
// InterfaceAddrs defines the interface used for retrieving network addresses.
type InterfaceAddrs func() ([]net.Addr, error)