Basic rounding tips

Post Reply
User avatar
demonnic
Posts: 884
Joined: Sat Dec 05, 2009 3:19 pm

Basic rounding tips

Post by demonnic »

This came up briefly in #mudlet today, and sparked off a quick bit of coding and discussion, and I figured I would share the outcome of all this with the forums. Nothing mind blowing or earth shattering here, but hoping to cover a few of the situations one could conceivably run in to.

Situation #1:
I just want to drop the fractional part of a number, I don't care about rounding.

Solution #1A
You can call math.floor(number) or math.ceil(number). Floor drops the fractional portion, whereas ceil rounds up to the next whole integer. In neither of these operations are any rounding rules followed.

Solution #1B
You can use string.format("%d", number) and it will essentially behave the same way as math.floor() does

Situation #2
I want to round a number to the next whole integer and follow basic rounding rules (.5 or higher rounds up, .anything lower rounds down)

Solution #2A
math.floor(number + 0.5)
This is a real basic cheat. If you add .5 to a number and it still isn't the next integer up, it should be rounded down... so math.floor handles it in either case

Solution #2B
string.trim(string.format("%99.0f", number))
This will format the number with 0 digits on the fractional side of the decimal point, and up to 99 characters long. It will also front load the digit with spaces to fill out those fields, so I've used string.trim to eliminate those. This isn't in Lua 5.1 normally, but we've included it with mudlet's lua.

Situation #3
I know exactly how I want my number formatted

Solution #3
The easiest thing to do here is just to use string.format's %f formatter. The number before the decimal is how many total characters to display in, and the number after the decimal is how many fractional digits to keep. As an example, string.format("%8.2f", 123.257) will return " 123.26" as there are 5 numbers and a decimal point, which leaves 2 spaces required to make up the 8 total characters. the formatter will do the rounding for you though

Situation #4
I'm some sort of nerd and want to round a number to a certain number of significant digits

Solution #4

I have written a function! round(numberToRound, numberOfSignificantDigits). So round(17246.18374, 3) would return 17200, with 6 significant digits it would return 17246.2, etc.
Code: [show] | [select all] lua
function round(number, sigDigits)
  local decimalPlace = string.find(number, "%.")
  if not decimalPlace or (sigDigits < decimalPlace) then
    numberTable = {}
    count = 1
    for digit in string.gmatch(number, "%d") do
      table.insert(numberTable, digit)
    end
    local endNumber = ""
    for i,digit in ipairs(numberTable) do
      if i < sigDigits then
        endNumber = endNumber .. digit
      end
      if i == sigDigits then
        if tonumber(numberTable[i + 1]) >= 5 then
          endNumber = endNumber .. digit + 1
        else
          endNumber = endNumber .. digit
        end
      end
      if i > sigDigits and (not decimalPlace or (i < decimalPlace)) then
        endNumber = endNumber .. "0"
      end
    end
    return tonumber(endNumber)      
  else
    local decimalDigits = sigDigits - decimalPlace + 1
    return tonumber(string.format("%" .. decimalPlace - 1 .. "." .. decimalDigits .. "f", number))
  end
end

User avatar
Akaya
Posts: 414
Joined: Thu Apr 19, 2012 1:36 am

Re: Basic rounding tips

Post by Akaya »

Awesome :)

Post Reply