Pandas : Add 2 time columns and create a new column

Question

I am new to Pandas and was wondering if this is possible.

I have two columns one with epoch time and another with milliseconds count. I want to create a 3rd column, that has time in milliseconds using both of these as single time column that has time in ms, so that I can easily select data frame between given times. Could someone please help me with this.

my_time         my_ms_counts
1500702208      1
1500702208      2
1500702208      3
1500702208      4
1500702208      5

With these 2 I want a single time column.

Answer 1

pd.to_datetime will convert your epoch time to a date (specify seconds for the unit), then use pd.to_timedelta to add the milliseconds.

df['datetime'] = (pd.to_datetime(df['my_time'], unit='s') 
                  + pd.to_timedelta(df['my_ms_counts'], unit='ms'))

#      my_time  my_ms_counts                datetime
#0  1500702208             1 2017-07-22 05:43:28.001
#1  1500702208             2 2017-07-22 05:43:28.002
#2  1500702208             3 2017-07-22 05:43:28.003
#3  1500702208             4 2017-07-22 05:43:28.004
#4  1500702208             5 2017-07-22 05:43:28.005

Alternatively you can combine both columns after converting to a similar unit. Choose the smaller unit, ms, to avoid issues with imperfect float64 representations.

df['full_time_ms'] = df['my_time']*1000 + df['my_ms_counts']

#      my_time  my_ms_counts                datetime   full_time_ms
#0  1500702208             1 2017-07-22 05:43:28.001  1500702208001
#1  1500702208             2 2017-07-22 05:43:28.002  1500702208002
#2  1500702208             3 2017-07-22 05:43:28.003  1500702208003
#3  1500702208             4 2017-07-22 05:43:28.004  1500702208004
#4  1500702208             5 2017-07-22 05:43:28.005  1500702208005

Now you could convert that single column specifying ms

pd.to_datetime(df['full_time_ms'], unit='ms')

#0   2017-07-22 05:43:28.001
#1   2017-07-22 05:43:28.002
#2   2017-07-22 05:43:28.003
#3   2017-07-22 05:43:28.004
#4   2017-07-22 05:43:28.005
#Name: full_time_ms, dtype: datetime64[ns]