在项目开发中,时常会需要将类似邮件发送、定时的客户端消息推送等需求扔给后端多线程任务处理系统来处理, Sidekiq就是其中一款后台异步任务处理服务,因为这个系统使用的是Redis,所以安装前需要先将Redis安装到系统中。

安装

# 安装 Redis
# Gemfile
gem 'sidekiq', '~> 3.4.2'
gem 'redis', '~> 3.2.1'
group :development do
  gem 'capistrano-sidekiq'
end
# config/application.rb
module YourApp
  class Application < Rails::Application
    config.active_job.queue_adapter = :sidekiq
  end
end

配置

将redis配置信息写到secrets.yml中

redis: &redis
   redis_server: 'localhost'
   redis_port: 6379
   redis_db_num: 0
   redis_namespace: 'highlander_sidekiq'


development:
  <<: *redis

在 application.rb 最后一行加入:

CONFIG = Rails.application.secrets

initializers 下新建 sidekiq.rb,用来初始化redis和sidekiq的配置

redis_server = CONFIG.redis_server
redis_port = CONFIG.redis_port
redis_db_num = CONFIG.redis_db_num
redis_namespace = Rails.application.secrets.redis_namespace


Sidekiq.configure_server do |config|
  p redis_server
  config.redis = { url: "redis://#{redis_server}:#{redis_port}/#{redis_db_num}", namespace: redis_namespace }
end

Sidekiq.configure_client do |config|
  config.redis = { url: "redis://#{redis_server}:#{redis_port}/#{redis_db_num}", namespace: redis_namespace }
end

可以把参数写入yml文件中,供cli方式启动时使用

# config/sidekiq.yml
# 并发数
:concurrency: 5
:pidfile: tmp/pids/sidekiq.pid

# 队列分类,例如高低优先级等等
:queues:
    - default
    - [lowqueue, 2]
    - [highqueue, 2]

development:
  :concurrency: 5
staging:
  :concurrency: 10
production:
  :concurrency: 20

配合ActiveJob使用

# 新建任务
rails generate job Example
# app/jobs/wish_remind_job.rb
class ExampleJob < ActiveJob::Base
  # 指定队列分类
  queue_as :default

  def perform(wisher, wished_user)
    puts "用户#{wisher.id}向你许愿"
  end
end
# cli方式启动sidekiq
bundle exec sidekiq -C config/sidekiq.yml
# app/v1/wishes.rb
module V1
  class Wishes < Dispatch
    ...
    post do
      ...
      # 加入到队列并立即执行
      WishRemindJob.perform_later @current_user
      ...
    end
  end
end
# 加入到队列并立即执行
MyJob.perform_later record
# 加入队列,并指定在明天中午执行
MyJob.set(wait_until: Date.tomorrow.noon).perform_later(record)
# 加入队列,并延迟在一星期后执行
MyJob.set(wait: 1.week).perform_later(record)

可用的回调

  • before_enqueue
  • around_enqueue
  • after_enqueue
  • before_perform
  • around_perform
  • after_perform
class GuestsCleanupJob < ActiveJob::Base
  queue_as :default

  before_enqueue do |job|
    # do something with the job instance
  end

  around_perform do |job, block|
    # do something before perform
    block.call
    # do something after perform
  end

  def perform
    # Do something later
  end
end

异常处理

class GuestsCleanupJob < ActiveJob::Base
  queue_as :default

  rescue_from(ActiveRecord::RecordNotFound) do |exception|
   # do something with the exception
  end

  def perform
    # Do something later
  end
end

Capistrano集成

# Capfile
require 'capistrano/sidekiq'
# to require monit tasks, Only for capistrano3
require 'capistrano/sidekiq/monit'

There is a known bug that prevents sidekiq from starting when pty is true on Capistrano 3.

set :pty,  false

有关sidekiq的命令,使用cap -T sidekiq查看

cap sidekiq:quiet                  # Quiet sidekiq (stop processing new tasks)
cap sidekiq:respawn                # Respawn missing sidekiq proccesses
cap sidekiq:restart                # Restart sidekiq
cap sidekiq:rolling_restart        # Rolling-restart sidekiq
cap sidekiq:start                  # Start sidekiq
cap sidekiq:stop                   # Stop sidekiq

cap-sidekiq的参数配置及默认值

:sidekiq_default_hooks =>  true
:sidekiq_pid =>  File.join(shared_path, 'tmp', 'pids', 'sidekiq.pid')
:sidekiq_env =>  fetch(:rack_env, fetch(:rails_env, fetch(:stage)))
:sidekiq_log =>  File.join(shared_path, 'log', 'sidekiq.log')
:sidekiq_options =>  nil
:sidekiq_require => nil
:sidekiq_tag => nil
:sidekiq_config => nil
:sidekiq_queue => nil
:sidekiq_timeout =>  10
:sidekiq_role =>  :app
:sidekiq_processes =>  1
:sidekiq_concurrency => nil
# config/deploy.rb

set :sidekiq_concurrency, 5
set :sidekiq_queue, ['default,3']
## or use config.yml file
set :sidekiq_config, "#{current_path}/config/sidekiq.yml"

capistrano-sidekiq默认会添加一些hooks并在capistrano-rails部署的时候触发, 如果想手动操作,可以将sidekiq_default_hooks设置为false

task :add_default_hooks do
  after 'deploy:starting', 'sidekiq:quiet'
  after 'deploy:updated', 'sidekiq:stop'
  after 'deploy:reverted', 'sidekiq:stop'
  after 'deploy:published', 'sidekiq:start'
end

监控

命用自带的监控页面

# Gemfile
gem 'sinatra', :require => nil

# config/routes.rb
require 'sidekiq/web'
mount Sidekiq::Web => '/sidekiq'

# view page
<%= link_to 'Monitoring', sidekiq_web_path %>

参考