Query Objects

I’m trying to migrate my models to be dumb data bags that have some useful state altering methods. I’ve found that using query objects, I was able to reduce the complexity of some of my models and controllers.

The Problem

My User model is gigantic. It’s well over 600 LOC, and it’s growing at a scary rate. Testing is becoming a nightmare because of all the methods. A few methods I have realized, I could remove completely and extract them into query objects.

app/models/user.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14

class User < ActiveRecord::Base
  # ...

  def self.search text
    where {
      (users.email.like "%#{text}%") |
      (users.username.like "%#{text}%") |
      (users.first_name.like "%#{text}%") |
      (users.last_name.like "%#{text}%")
    }
  end

  # ...
end

Now this method isn’t exactly all that complicated but my User model is rittled with these methods. They are just there to be used by the controller like so:

app/controllers/users_controller.rb
1
2
3
4
5

class UsersController < AuthorizedController
  def index
    @users = User.search(params[:q]).page(params[:page])
  end
end

All that method does, is execute a query. It doesn’t alter the state of any model. This could be extracted out into its own class and be isolated enough that testing becomes very simple.

The Solution

The solution is simple, kill The Batman. First, I need a query class. When creating query classes, keep in mind that you need to interact with ActiveRecord::Relation objects. This will allow for the query objects to be chained and used again later.

app/queries/user_search_query.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

class UserSearchQuery
  # @param [ActiveRecord::Relation] relation
  def initialize relation=User.scoped
    @relation = relation
  end

  # @param [String] text
  def search text
    @relation.where {
      (users.email.like "%#{text}%") |
      (users.username.like "%#{text}%") |
      (users.first_name.like "%#{text}%") |
      (users.last_name.like "%#{text}%")
    }
  end

  # @param [String] text
  def self.search text
    new.search(text)
  end
end

Second I need to replace the call to User.search from the UsersController

app/controllers/users_controller.rb
1
2
3
4
5

class UsersController < AuthorizedController
  def index
    @users = UserSearchQuery.search(params[:q]).page(params[:page])
  end
end

Now looking at this, it is easy to say, “Well that didn’t change much”, but it did. I can remove the old method from the User model, and reduce the complexity a little bit by doing this.

Chaining

Here is an example of chaining some queries together

app/queries/new_users_query.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

class NewUsersQuery
  # @param [ActiveRecord::Relation] relation
  def initialize relation=User.scoped
    @relation = relation
  end

  def today date=DateTime.now
    between(date.beginning_of_day, date.end_of_day)
  end

  def last_seven_days
    date = DateTime.now.begninning_of_day
    between(date - 7.days, date.end_of_day)
  end

  def between starting, ending
    @relation.where {
      (users.created_at >= starting) &
      (users.created_at < ending)
    }
  end
end

Now this may sound a little dumb, but this is merely just an example.

1
2
3
4

query = UserSearchQuery.new(NewUsersQuery.today)
query.search('someone').each do |user|
  NotificationMailer.welcome(user).deliver
end

References

refactoring, ruby

Ruby Mixin Exploration

Recently I’ve been playing with Ruby’s class_eval and define_method to append methods to an object. Meta programming is something new for me and I had a problem that would benefit from using some of Ruby’s cool features.

The Problem

There was a model that had 6 fields that needed to be generated at some point in time. Some fields had to be unique and others had to be generated a bit differently. Here is a contrived example of what I was doing:

Examples
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

class User < ActiveRecord::Base
  def generate_api_token
    begin
      self.api_token = SecureRandom.urlsafe_base64(32)
    end while self.class.exists?(api_token: self.api_token)
    self.api_token
  end

  def generate_new_password
    self.password = SecureRandom.urlsafe_base64(16)
  end
end

class Widget < ActiveRecord::Base
  def generate_slug
    begin
      self.slug = SecureRandom.urlsafe_base64(32)
    end while self.class.exists?(slug: self.slug)
    self.slug
  end
end

If these generators are spread across multiple models, it will cloud the source code and a test will need to be written for every generator. If we utilized shared examples, there wouldn’t be a need to write a test for every single generator.

The Goal

In the end, I wanted an easy to read DSL. Adding more DSL to a class can mean a ”code smell”, however in this case, I do not think it is a code smell.

app/models/some_model.rb
1
2
3
4
5
6
7
8
9
10
11

# End goal
class SomeModel < ActiveRecord::Base
  include Generatable

  generatable :some_field,   lambda { SecureRandom.hex(32) }
  generatable :unique_field, lambda { SecureRandom.hex(32) }, unique: true
end

s = SomeModel.new
s.generate_some_field #=> some 32 character string
s.generate_unique_field #=> some 32 character unique string

The pros

  • Easy to read
  • Easy to figure out where generatable is being loaded from
  • Compact
  • Testing becomes easier (more on this later)

The cons

  • Can’t use the model in the lambda
  • Adds to model complexity
  • Lose some control on testing (more on this later)

Solution

This is a perfect case where ActiveSupport::Concern applies. This is functionality that multiple models can utilize and thus, should be put into a mixin and included to those models. My initial implementation only made a simple generator but, it does show how class_eval works and is an easy example to grok.

app/models/concerns/generatable.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22

# Partially what I wanted
module Generatable
  extend ActiveSupport::Concern

  module ClassMethods
    # @param [String] field the name of the field you want to generate
    # @param [Proc] generator the lambda or Proc that you wish to use as the
    #   generator
    # @param [Hash] options
    # @option options :unique
    # @return [void]
    def generatable field, generator, options={}
      field = field.to_s
      class_eval do
        define_method "generate_#{field}" do
          self.send("#{field}=", generator.call)
        end
      end
    end
  end

end

The next requirement I had to fulfill was generating unique values. This took a little bit more thought.

app/models/concerns/generatable.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

module Generatable
  extend ActiveSupport::Concern

  module ClassMethods
    # @param [String] field the name of the field you want to generate
    # @param [Proc] generator the lambda or Proc that you wish to use as the
    #   generator
    # @param [Hash] options
    # @option options :unique
    # @return [void]
    def generatable field, generator, options={}
      field = field.to_s

      if options[:unique]
        class_eval do
          define_method "generate_#{field}" do
            begin
              self.send("#{field}=", generator.call)
            end while self.class.exists?(field.to_sym => self.send(field))
            self.send(field)
          end
        end
      else
        class_eval do
          define_method "generate_#{field}" do
            self.send("#{field}=", generator.call)
          end
        end
      end
    end
  end

end

This isn’t exactly pretty. Infact, one could argue that this is a bit difficult to follow due to the if options[:unique] statement. Another possible solution could be to break this up into three methods like so:

app/models/concerns/generatable.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50

module Generatable
  extend ActiveSupport::Concern

  module ClassMethods
    # Appends a generator
    # @param [String] field the name of the field you want to generate
    # @param [Proc] generator the lambda or Proc that you wish to use as the
    #   generator
    # @param [Hash] options
    # @option options :unique
    # @return [void]
    def generatable field, generator, options={}
      if options[:unique]
        generatable_unique field, generator
      else
        generatable_simple field, generator
      end
    end

    # Append a unique generator
    # @param [String] field the name of the field you want to generate
    # @param [Proc] generator the lambda or Proc that you wish to use as the
    #   generator
    def generatable_unique field, generator
      field = field.to_s
      class_eval do
        define_method "generate_#{field}" do
          begin
            self.send("#{field}=", generator.call)
          end while self.class.exists?(field.to_sym => self.send(field))
          self.send(field)
        end
      end
    end

    # Append a simple generator
    # @param [String] field the name of the field you want to generate
    # @param [Proc] generator the lambda or Proc that you wish to use as the
    #   generator
    def generatable_simple field, generator
      field = field.to_s
      class_eval do
        define_method "generate_#{field}" do
          self.send("#{field}=", generator.call)
        end
      end
    end
  end

end

Testing

Earlier I mentioned that this makes testig easier and then followed up with a loss of control in the tests. The Generatable module will need to be tested in some fashion, whether it is directly testing it via including it on a dummy model or making a shared example where the test would have it_behaves_like somewhere.

Testing a module directly:

Module Testing
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19

module Say
  extend ActiveSupport::Concern
  def hello
    "hello"
  end
end

class DummyClass
  include Say
end

describe Say do
  let(:dummy_class) { DummyClass }
  let(:dummy) { dummy_class.new }

  it "get hello string" do
    dummy_class.hello.should == "hello"
  end
end
spec/support/shared_examples/generatable_fields.rb
1
2
3
4
5
6
7
8
9
10
11

shared_examples 'generatable fields' do |field|
  describe "#generate_#{field}" do
    subject { described_class.new }

    it "should change #{field}" do
      expect {
        subject.send("generate_#{field}")
      }.to change(subject, field.to_sym)
    end
  end
end
spec/models/user_spec.rb
1
2
3
4

describe User do
  include_examples 'generatable fields', :api_token
  include_examples 'generatable fields', :password
end

Resources

rails, rspec, ruby, testing

No Internet Means Better Test Isolation

One day I found myself riding in a car with my family and it was a long road trip, with little to no internet and realized that my tests required internet connectivity. I was also heavily dependent upon code examples that I could find.

I’ve been absolutely horrible with testing my code. I would write code, test it out in the ruby console and then push it live. It felt like I was moving fast. I was being agile. Wrong! Testing is necessary in order to succeed. Often I would push code and do the good ole’ cowboy code em’ up. This is a horrible idea, don’t do that.

Stubs And Mocks

I’ve seen arguments for and against the use of stubs and mocks. I like mocks and stubs. I do think they are useful, and they have their place. Many people find themselves over stubbing and over mocking. This can lead to brittle tests.

I suppose it is tempting, if the only tool you have is a hammer, to treat everything as if it were a nail. Abraham H. Maslow

Corey Haines has a good presentation called Yay Mocks!. Definitely worth the time to watch.

Using stubs and mocks in a restrained fashion will result in maximum payoff. I attempted to stub and mock everything in a controller unit test. This turned out to be a hellish nightmare to maintain. I did discover some cool logical control flow that I could utilize. I could force CanCan to raise exceptions and cause some fun errors in the controller.

spec/controllers/threads_controller.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51

describe ThreadsController do

  describe '#create' do
    context 'when a user is signed in' do
      before { sign_in user }
      context 'and is authorized' do
        before { controller.stub(:authorize!).and_return(true) }
        context 'and the thread is created' do
          before do
            post :create, thread: {title: 'Some Title'}
          end
          it { should respond_with 302 }
          it { should set_the_flash[:success] }
          it 'should create the thread' do
            expect(Thread.count).to eq(1)
          end
        end

        context 'and the thread is not created' do
          before do
            Thread.any_instance.stub(:valid?).and_return(false)
            post :create, thread: {title: 'Some Title'}
          end
          it { should respond_with 200 }
          it { should set_the_flash[:error] }
          it 'should not create a thread' do
            expect(Thread.count).to eq(0)
          end
        end
      end

      context 'and is not authorized' do
        before do
          controller.stub(:authorize!).and_raise(CanCan::AccessDenied)
          post :create, thread: {title: 'Some Title'}
        end
        it { should respond_with 403 }
        it { should set_the_flash[:error] }
        it 'should not create a thread' do
          expect(Thread.count).to eq(0)
        end
      end
    end

    context 'when a user is not signed in' do
      before { post :create, thread: {title: 'Some Title'} }
      it { should respond_with 403 }
    end
  end

end

Testing permissions should not apply to controller tests. This ia a boundary and should be stubbed. You can unit test your permissions in another set of tests, but they do not belong in the controller test.

Boundaries

I strongly urge you to watch Boundaries by Gary Bernhardt. Understanding where an object’s boundaries lie, is key for test isolation. Ruby has a nice open class principle that allows for stubing and mocking. Earlier I pointed out that in a controller permissions were considered a boundary.

No Internet

Now for the fun story. I had no internet on my laptop while we traveled down the highway. I was writing code and then realized I should run my tests real quick to ensure I didn’t cause a massive problem. What I found out to my surprise is that more than half of our test suite failed, due to the need to talk to a 3rd party API. Ooops.

All I had was my phone and even then all I had was spotty coverage. I remembered seeing the two videos mentioned above, and thought to myself, “This is similar to what TDD is right?” I was right, as I went through and looked at why the tests were failing, I noticed that they were failing in places that were difficult to test. On top of the areas difficult to test, it was often over stepping boundaries.

I used my phone to look up RSpec Mocks and tried out a few examples to make sure I understood what was going on.

1
2
3
4
5
6

foo = double('User')
foo.stub(:update_attributes).and_return(true)

it 'should return true' do
  expect(foo.update_attributes({:garbage => 'in'})).to eq(true)
end

Revelation

And when the test greened up, a sudden grin appeared on my face. I realized that I could do anything. I have never once been happy about testing in Java or PHP and I couldn’t help but be really excited when I started seeing some tests come up green.

I started to stub and mock places where I was making calls to an API. Many of these tests, I simulated errors that could be encountered while interacting with the API.

I realized that there were parts of our application that were extremely difficult to isolate. This was indicative of ”code smell”. I wrote some tests on how I wanted the peices to interact, then made the peices interact to make the tests green.

Being able to simulate errors from the API was a HUGE help. It allowed me to see potential errors and handle them appropriately rather than show a big ole’ 500 error page to our customer.

Resources

Here is a list of resources that I utilize all the time. In fact, they are bookmarked and I often visit them.

rails, rspec, ruby, testing

Stand Up Desk DIY

I have been hard at work over my birthday to now. I’ve spent the better part of 20 hours with my dad working on my new standing desk. Where I was working before was extremely cluttered and an upgrade to my life was needed.

I wasn’t able to take pictures from the initial start of the project, due to us having to do multiple things at once before Home Depot closed. However, after we settled down, I had Amanda come out and take pictures as we were working on the desk.

We built the desk using Oak tongue and groove boards. My parents finished an interior remodel of their house about two years ago. A lot of oak boards were left over and my dad suggested we use it instead of buying a new sheet of Birch.

I hate glossy polyurethane. Even a satin finish still bothers me some. For me wood isn’t supposed to be reflective. I opted for a Matte finish with a Dark Walnut stain.

Staining was probably the best feeling in the world. I was so close to being done!

After three layers of polyurethane and some serious tender love and care. I took it home with me to assemble! This table is super heavy due to all the Oak that we used.

Feels great now that I have this completed!

diy, health

I Graduated

Finally! I have graduated from the University of Texas at San Antonio with a Bachelor’s degree in Computer Science. I waited for 4.5 years (9 semesters) to finally reach this point in my life. Though now I wonder what awaits me in the future.

Now I’m able to work Full Time at ZippyKid where I help build and maintain the deployment system. Ever since I started working here in January 2012, I’ve grown a lot and I have learned a TON.

Things I’ve learned:

  • Rails is not the answer to all questions
  • Venturing out of your comfort zone is great
  • Good OO design is not easy
  • There is no silver bullet
  • No one really knows what they are doing in software engineering
achievement, school

Organizing Delayed Jobs

I despise consuming 3rd party APIs for one reason, speed. Many of the APIs I have to consume on a daily basis, respond slower than it would take to deliver an email. But, their integration with our application is absolutely critical. They are so critical infact, that if an error should happen during any point of the commissioning, then I need to be notified and or the job needs to be reattempted at a later date.

On many occassions, I have simply split the 3rd party API object into its own Rails model that is persisted in the database with a state_machine or some sort of identifier on it that lets the application know if it is in an erroneous state.

Billing is a prime example of this:

app/models/subscription.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20

#  Table: subscriptions
#    - user_id
#    - subscription_id
class Subscription
  belongs_to :user

  def subscribe
    # subscribe to api here
    # set subscription_id = id in api
  end

  def unsubscribe
    # unsubscribe from api here
    # invoke .destroy
  end

  def update_billing_information
    # interact with api here
  end
end

I mentioned earlier that I would use a state machine on models that interact with APIs. However, in this example, if the subscription_id is not set, then the Subscription will be in an erroneous state.

I prefer to make job classes, only because they allow for me to separate my backgrounding code, from my model code. Plus, there is the possibility that you are sticking too much data into the delayed job queue by doing User.delay.send_some_email or @user.delay.send_some_email is even worse. It will turn that ruby object into a YAML formatted object and stick it into the queue.

The following is the best way, IMO to do backgrounding with Delayed Job. Just put a few simple values in and it keeps the queue lean.

app/jobs/subscribe_subscription.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14

class SubscribeSubscription
  def initialize id
    @id = id
  end

  def perform
    subscription = Subscription.find(@id)
    subscription.subscribe
  end

  def error job, exception
    puts "[ERROR] subscription=#{@id} message=#{exception.message}"
  end
end

I would then invoke the Delayed::Job queue by using an observer. Now this part you don’t have to necessarily put into an observer, but it makes for a cleaner model. As an aside, I store my observers in app/observers, it just makes it a little easier to organize my thoughts.

app/observers/subscription_observer.rb
1
2
3
4
5

class SubscriptionObserver < ActiveRecord::Observer
  def after_create subscription
    Delayed::Job.enqueue(SubscribeSubscription.new(subscription.id))
  end
end

We would then be able to do the following:

app/controllers/subscriptions_controller.rb
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21

class SubscriptionsController < AuthorizedController
  # ...

  def create
    sub  = current_user.subscriptions.build(subscription_params)

    if sub.save
      # success
    else
      # booo failure
    end
  end

  # ...

  private

  def subscription_params
    params.require(:subscription).permit(:name, :more, :fields)
  end
end

We have decoupled the 3rd party interaction from the controller and put it into a background job, where it will be handled. However, in this straw man example I did not show a proper way to handle failure. If I was going to actually do this with a payment system, I would definitely notify the user that something is up and make them reattempt the sign up again.

background jobs, ruby

Flying With Php

I discovered the Flight micro-php framework this week. I found it because I was looking for a simple PHP 5.3+ framework that didn’t have a large amount of extra cruft. I had a project to do for my Database class at UTSA where we had to utilize an Oracle database. Not exactly my choice for databases, but we had to make do with what we were given.

It is a very sparse framework that looks very similar to Sinatra except it is even lighter. Flight provided me ways to easily extend and customize.

I used a fairly straight forward file structure:

1
2
3
4
5
6
7

|-assets
|-config
|-controllers
|-db
|-lib
|-models
|-views

Flight doesn’t come with a Database class. So you will have to roll your own or use an ORM or use Mike Cao’s Sparrow library. I haven’t had a chance to play with it yet, but it looks very promising. Unfortunately, it does not support Oracle, so I had to roll my own implementation, and it’s not pretty by any means.

I look forward to playing with this library some more, and hopefully building something with it soon.

Here is my project if you want to know more. It was built for a school project so keep that in mind while you are looking at it in horror.

php, programming

Build It

I am going to entertain you with an anecdote or two. I was recently talking with one of my friends about what they should do for their next side project. They didn’t know what to do or where to start. The only real sage advice I could give them was to:

Build something. Learn from it, reflect on what you just built and work on another project

I have been doing this for so long that it’s almost second nature. I hardly think about the process anymore. I’ve started so many side projects that have failed miserably, but I have learned a trick or two that I applied to other on-going projects.

I’ve gotten hung up on few of my projects to the point I couldn’t solve the issue at hand. I didn’t know what to do and thought I needed to take a break to work on one of my side projects Starship Gunner. I was playing around with how to organize my background jobs and setting up a decent test suite that I had an epiphany. For some reason, the problem I was hung up on just made sense.

I remember sitting in my Advanced Programming class and not understanding what the hell * and & meant in C. I thought it was all just heresy and voodoo magic. That same semester I ran my car out of oil and had to pull my engine and replace parts. Mind you, I had no prior experience with cars. I just started hacking away at the problem and then got the motor put back into the car.

When that happened, I realized that I could do anything I wanted to. I can build any application that I wanted to. I could put any parts on my car that I wanted to. Suffice to say that I could put any bit of code on my applications. And the * and & weren’t enigmas anymore. I picked up on pointer arithmetic pretty quickly and things started working out.

TL; DR. I rebuilt the engine on my 1972 BMW. I realized that I can apply the knowledge I gained from that project to any project I encounter. You should try this approach, it’ll pay off.

motivation